The Cost of Bucking College Rankings

By Michele Tolela Myers
Sunday, March 11, 2007; B07

The Washington Post

Like most college presidents, I have seen many prospective students and their parents show up on campus in recent months, clutching their well-worn copies of U.S. News & World Report's rankings issue. U.S. News has smartly tapped into students' need to sort out colleges and universities in a rational way. Parents, who face increasing college costs, understandably want to know where best to make that expensive investment.

U.S. News benefits from our appetite for shortcuts, sound bites and top-10 lists. The magazine has parlayed the appearance of unbiased measurements into a profitable bottom line.

The problem is that the U.S. News college rankings are far from reliable.

Turns out that some of their numbers are made up. I know that firsthand. Two years ago, we at Sarah Lawrence College decided to stop using SAT scores in our admission process. We didn't make them optional, as some schools do. We simply told our prospective students not to bother sending them. We determined that the best predictors of success at Sarah Lawrence are high school grades in rigorous college-prep courses, teachers' recommendations and extensive writing samples. We are a writing-intensive school, and the information produced by SAT scores added little to our ability to predict how a student would do at our college; it did, however, do much to bias admission in favor of those who could afford expensive coaching sessions.

Since we dropped the SAT altogether, we no longer provide SAT information to U.S. News & World Report. Our two years' experience with this practice has been very good. Faculty members report that our students continue to be terrific. Their average high school grades, high school ranks and grades in Advanced Placement courses have not changed.

But this principled decision has put us in jeopardy. I was recently informed by the director of data research at U.S. News, the person at the magazine who has a lot to say about how the rankings are computed, that absent students' SAT scores, the magazine will calculate the college's ranking by assuming an arbitrary average SAT score of one standard deviation (roughly 200 points) below the average score of our peer group.

In other words, in the absence of real data, they will make up a number. He made clear to me that he believes that schools that do not use SAT scores in their admission process are admitting less capable students and therefore should lose points on their selectivity index. Our experience, of course, tells us otherwise.

But the story does not stop here. When I reported this conversation at Sarah Lawrence, several faculty members and deans suggested that perhaps it was time to stop playing ranking roulette and opt out of the survey. A few colleges explore this option each year, but most don't follow through (Reed College is one of the few exceptions I know of), because, like unilateral disarmament, unilateral withdrawal from the U.S. News ranking system is dangerous.

We discovered how dangerous it can be through a presentation U.S. News made at the 2006 meeting of the North East Association for Institutional Research. There, the magazine indicated that if a school stops sending data, the default assumption will be that it performs one standard deviation below the mean on numerous factors for which U.S. News can't find published data. Again, making up the numbers it can't get.

The message is clear. Unless we are willing to be badly misrepresented, we had better send the information the magazine wants. We haven't yet decided what we will do. But if we don't go along, we understand we will be harmed because many students will assume that Sarah Lawrence is much less selective than it actually is.

The reality is that the magazine's rankings issue has a large circulation and that parents and students rely on these rankings to make a college choice that has enormous educational and financial implications. This gives the magazine the power to keep colleges playing the game it sets and controls.

Why should we care if we lose our place in their rankings? Because ultimately, so many people take these rankings seriously. I would at least like to let them know how misleading the whole affair is.

The writer is president of Sarah Lawrence College.

View all comments that have been posted about this article.

Would U.S. News Make Up Fake Data?

It's not unusual for college presidents to complain about U.S. News rankings (at least out of the earshot of U.S. News editors). But on Sunday, the president of Sarah Lawrence College publicly charged that the magazine is preparing to publish made up, false data about her institution. Meanwhile, Inside Higher Ed has learned that 10 other liberal arts college presidents are preparing a letter to be sent to hundreds of college presidents proposing a new set of policies that might challenge the role of the rankings. The policy options include complete non-cooperation with U.S. News and refusing to fill out the “reputational” survey — which many educators deride as a “beauty contest” that is particularly lacking in substance, even though it represents 25 percent of the magazine's rankings formula.

In an op-ed in The Washington Post, Michele Tolela Myers, Sarah Lawrence's president, wrote that because her college no longer collects or examines SAT scores, U.S. News officials have said that the magazine will just assume that the average SAT would have been one standard deviation (about 200 points) below the average of Sarah's Lawrence's peers. “In other words, in the absence of real data, they will make up a number,” wrote Myers.

In an interview Sunday, the head of the U.S. News college rankings division acknowledged that he had told Myers of the magazine's plan to use the system she described in her article. But Robert Morse said that while he told her that was the plan, he also said that the magazine would “seriously” consider other approaches, which he declined to name. Myers, in an interview Sunday, said flatly that Morse had never said there was any other approach under consideration.

The dispute between Sarah Lawrence and U.S. News highlights more than just the rankings issue. The reason Sarah Lawrence is facing this problem is that the college — possibly alone among the many colleges that are dropping SAT requirements — won't even look at SAT scores any more. As more liberal arts colleges like Sarah Lawrence have dropped SAT requirements, the norm has been to go “SAT optional,” meaning that students are still welcome to submit the scores. Because a majority of applicants do so, SAT-optional colleges continue to have average SAT scores to report to U.S. News.

And that's where the dispute starts to point to potential problems with both SAT averages and U.S. News. When applicants learn that a college is SAT-optional, it doesn't take an 800 math score to figure out the statistically wise strategy. If your scores are at or above reported averages, submit them. Otherwise, don't. Not surprisingly then, many colleges that go SAT-optional experience both a surge in applications and an increase in their SAT averages ... and their U.S. News rankings go up.

Along comes Sarah Lawrence and it makes a decision that it doesn't believe in the SAT, so it doesn't want to look at scores — period. As the college explains in its FAQ for applicants: “Our recent decision to remove all standardized testing from the admission process reflects the college's emphasis on writing rather than testing. That's right: we no longer look at standardized test scores. Our students aren't ‘numbers,' and we'd rather not use standardized test numbers to select them.”

Sarah Lawrence has long asked for more writing from applicants than most colleges request and Myers said in the interview that the college didn't drop the SAT until it studied its data extensively and concluded that the SAT offered “nothing” in terms of predicting students' success. Myers said she was aware of the way other colleges drop the SAT but continue to collect scores and data from many applicants, but she questioned whether this was educationally sound. “I'm not going to play that game,” she said.

The U.S. News rankings that will appear this fall would be the first in which Sarah Lawrence will no longer have any SAT data to report. With that date looming, Myers and Morse met and that was where he outlined the magazine's plan. He said he could not think of another institution where he had to propose this approach because around two-thirds of applicants at SAT-optional colleges submit scores and U.S. News will count that average as long as the percentage exceeds 50 percent.

On Sunday, Morse said that he was reluctant to talk about Sarah Lawrence's complaints until he could confer with the top editors at U.S. News. But pressed, he said that Myers had left out “a key fact” from her op-ed in not saying that he had told her U.S. News was considering approaches besides just assigning the college a lower SAT average based on no real data.

Morse said he had told Myers that the approach of cutting the SAT score a standard deviation from the peer group average was “what we were going to do,” but he said he added that a new approach might also be considered. He said that as of today, no other approach had been determined, and that the plan remains the same, but that the discussions were “a process still going on.”

Morse said that he regularly speaks to groups of institutional researchers about the U.S. News methodology, including how the magazine handles situations where colleges do not provide some of the data the magazine seeks. He said that the magazine doesn't always use the approach of just lowering estimates by a standard deviation, and that the approach varies from category to category. “There are various procedures, and we've been very transparent,” he said.

Myers said that in her meeting with Morse, he expressed skepticism about the averages he gets from SAT-optional colleges, saying that he believed they were managing to inflate their averages. But on Sunday, Morse denied believing that. He said that colleges dropping the SAT and submitting the average scores of those who continue to send them would experience “a long term decline” because he expected their graduation rates to decline, especially as projected by their SAT scores, which is part of the U.S. News formula. So while Morse said he knew that it is “widely believed” that going SAT-optional helps in the rankings, he said that wasn't the case. (Morse's thinking is based, of course, on the assumption that the students who don't report SAT scores will have lower retention and graduation rates — something that colleges that have gone SAT-optional say is not the case.)

Bob Schaeffer, public education director of the National Center for Fair & Open Testing, praised Sarah Lawrence and said it was taking a principled stand. “The U.S. News policy of fabricating data for colleges which refuse to report average scores it is both unfair and unethical,” he said, adding that the “practice further undermines the credibility of the magazine's already widely criticized rankings.”

A Broader Challenge to the ‘Ranksteering'

As Myers took her criticism of the rankings public, other presidents are working behind the scenes to challenge U.S. News. In her op-ed, Myers noted that some on her campus wanted to completely disassociate the college from the magazine's rankings, but that she feared doing so would hurt the college more, as more inaccurate data about it might be included. While she said the college was absolutely committed to keeping its policy of ignoring the SAT, she said that ignoring U.S. News was “too big a risk” for the college to take alone. ( Reed College has taken that step for a number of years, although Morse noted that much of the information that goes into the rankings is public and the magazine gathers it for Reed, which does require the SAT or the ACT.)

A new effort is being organized to allow colleges to act in concert so that they fight back against the rankings. Some of the inspiration for this new effort came in January, when Lloyd Thacker, founder of the Education Conservancy, gave a presentation to the Council on Independent Colleges called “Ranksteering: Driving Under the Influence.” Thacker's organization argues that the college admission process has become divorced from educational values and “ranksteering” is his term for the impact of U.S. News and other rankings.

The presentation prompted a group of 10 presidents to work on a letter — currently circulating among them — outlining strategies that could be used to take on the rankings. The presidents' idea is to send the letter to all Council on Independent Colleges members (more than 570 institutions) to try to build a movement with the clout to be effective. Among the ideas in the letter: refusing to provide any help to U.S. News, refusing to fill out the “reputational” survey, pledging not to advertise any rankings they receive from U.S. News, and/or posting a prominent statement on their Web sites noting the dubious qualities of many ratings.

In an interview Sunday, Thacker confirmed that such a letter from presidents is in the works, but declined to discuss timing or to name the presidents involved, except to say that they are all from liberal arts institutions, that Myers of Sarah Lawrence had not been involved and that her op-ed was not part of this campaign. At the same time, he said that the issues Myers raised were completely consistent with the Education Conservancy's critiques and those of the presidents' letter.

Thacker said that was most striking about Sarah Lawrence's actions and the Myers op-ed was a willingness to stand up to both the College Board (as the sponsor of the SAT) and U.S. News. “Very few colleges feel they can act as singular moral agents,” Thacker said. “What she is doing is demonstrating the courage of her convictions.”

In the end, Thacker predicted that places like Sarah Lawrence would do well with applications, even if the U.S. News rankings include inaccurate information about them. Reed has made its rejection of U.S. News “its calling card,” Thacker said, and Sarah Lawrence may benefit from its ability “to distinguish itself by demonstrating its character in its admissions process.”

Morse, of U.S. News, said he didn't know anything about the letter circulating among college presidents. But he noted that there have been complaints about the rankings in the past that did not amount to much, and that law deans have been complaining about the law school rankings for years — even as the rankings continue to be popular with applicants. He also noted that much of the information in the rankings is also provided to the federal government so the magazine could still get it.

On the reputational survey, Morse said that 70 percent of liberal arts colleges currently fill it out. “It would have to fall a lot to be statistically significant,” he said. “If just 10 college presidents don't return the survey, it isn't one iota going to dilute the response rate.”

Morse noted that many of the presidents who don't like his ratings also oppose Education Secretary Margaret Spellings' attempts to make comparable data more widely available on various measures of student learning. He said he wasn't particularly worried about the fact that these presidents “are trying to derail us.”

Myers, the Sarah Lawrence president, said she feels good about her college's direction, and she noted that some of the strongest endorsements she has received for the SAT stance have come from high school counselors who like the message the college has sent to students.

As for the meaning of the dispute, Myers said: “It says a lot about U.S. News and higher education in general that we are willing to play dead most of the time just so this magazine can sell the one issue a year that makes a lot of money.”

— Scott Jaschik

Comments

US News Rankings

Hooray for Michele Myers! I wrote about this same topic in “Outflanking the Rankings” in the current issue of AGB's TRUSTEESHIP magazine —- see http://www.trinitydc.edu/about/president/012007_Trusteeship_magazine.php

At the very least, presidents and deans should refuse to fill out the bogus “reputational survey” that is a shameful excuse for assessing academic quality, and that has also fostered all kinds of tawdry conduct on the part of those who fill it out.

Instead, it's time for institutional leaders to take a true lead in establishing clear, transparent and effective performance reporting systems so that our prospective students and families can feel more confident that they have the best facts at hand in making the college choice.

We have the talent to do this; let's show some willpower!

Pat McGuire , President at Trinity (Washington) University, at 7:41 am EDT on March 12, 2007

Where is the quality control?

It is entirely in keeping with history that our postsecondary institutions would resist ranking of any kind, except if they were in charge.

It was 1911 when the US Department of Education (then known as the Bureau of Education) last tried to publish a list that stratified or ranked higher institutions according to how their college graduates performed in graduate school. This report was suppressed by both Pres. Taft and his successor, a professor himself, Pres. Wilson, at the behest of the institutions. (cf. David Webster in Hist. of Ed. Qtrly, 1984, 499-511.)

However, I still think that it is worth exploring the possibility of having U.S. News and World Report, or some other group that is independent of the accrediting guilds, conduct QA/QC audits of higher education. The blatant self-serving, incestuous system that we now have simply doesn't serve the public – instead, it now serves the institutions.

Glen S. McGhee , Dir., at Florida HIgher Education Accountability Project, at 8:45 am EDT on March 12, 2007

Rankings: A Love and Hate Relationship

I am encouraged by the discussion occurring around the issue of rankings.

However, at the same time, I do not expect much to change regarding how rankings are determined by the U.S. News or any of the other half dozen or so rank fabricating publications currently in existence. Why?

Simply put, I continue to hear college Presidents, department chairs, college deans and almost every university administrator gripe about college rankings when they are not in their favor, however, these same individuals proudly and loudly tout in alumni publications and through campus media sources their rankings when their program or school is favorably ranked.

Basically, university administrators are caught between a rock and a hard place. Deep down, they know that these rankings are largely meaningless, based on “physical looks” more so than quality of instruction. However, they also know that the uninformed general public places a great deal of weight on these rankings, thus they also rely on these “fabricated” rankings to help boost applications and the quality of student seeking enrollment in their prestigious institutions.

In short, until those universities continually in the top 50 in all rankings begin to come forward with concerns about this process, I am afraid that little known but quality institutions such as Sarah Lawrence College will continue to suffer under these fabricated rankings.

Kevin Leonard , Sr. Program Coordinator at Michigan State University, at 9:10 am EDT on March 12, 2007

Entire System is Flawed

Each department must be compared with the departments of other schools if rankings are to have value. The attempt to rank an entire school on entrance exams for all students is absurd. It is amazing that the keepers of our intelligence have suffered this system for as long as they have.

William Sumner Scott, J.D.

Judicial Equality Foundation, Inc.

wss@jefound.org

William Sumner Scott, J.D. , at 9:55 am EDT on March 12, 2007

Praise for Sarah Lawrence

Good stand by Sarah Lawrence and hope others will follow suit. If colleges and universities supplied data to the American Association of Universities and Colleges wouldn't it be possible for that group in consultation with the schools to come up with some agreed upon ranking criteria?

Patrick Mattimore , Teacher, at 9:55 am EDT on March 12, 2007

Michelle Myers — Whining Cry baby

Please! Michelle Myers — on behalf of low-ranking Sarah Lawrence doesn't want to submit standard data to US News for comparative purposes — and then complains that Sarah Lawrence isn't being evaluated fairly?

Ms. Myers would do well to see past her own bruised ego and recognize that US News college ranking aren't intended for her agrandizement — they are intended for families of college bound students to sort out the differences between institutions of higher education. By refusing to submit SAT data, Myers is obfuscating a critical tool in the college planning process for families.

Rather than ‘faking' data, US News is making a responsible decision to make a reasonable substitution of data for comparative purposes.

US News & World Report has always been the gold standard for fair, tell-it-like-it-is reporting. It is absurd for a whining, cry baby college president of a third tier college to suggest that US News would risk it's reputation to discredit Sarah Lawrence. Please!

Susan A. Patton , at 9:55 am EDT on March 12, 2007

Will the leaders of higher ed stand up?

As a college counselor I find it deploarable that colleges are penalized in their rankings. Education is a public trust and it does not serve this trust to fabricate scores nor penalize universities for taking the challenge of education by individuating their admission process.

Although I would be happy for USNEWS to stop ranking, if they must rank, I want them to do so ethically. If they are fabricating numbers in one part of their magazine who is to say they are not doing so in another? Let us be clear: It is fraud.

USNEWS states that they rank colleges “to help you [students] make one of the most important decisions of your life.” And it sells magazines. A lot of them. It has generated an entire revenue model on the internet. They say that “to find the right college, you need a source of reliable and consistent data-information that lets you compare one college with another and find the differences that matter to you.” They are failing in this mission and need to take responsibility for it.

If USNEWS is incapable of doing this, I wonder what would happen if 3500 + universities, their professors and the thousands of high schools that subscribe to USNEWS simply canceled their subscriptions.

Shaun mcElroy , Counselor, at 10:10 am EDT on March 12, 2007

Name-Calling

Susan A. Patton's comment above amounts to nothing but name-calling. It also has the appearance of having a hidden agenda: Does Ms. Patton have a personal or perhaps financial interest in running down Sarah Lawrence College & singing the praises of US News' college rankings. Does she, in fact, have any relationship to US News?

I thought overt ad hominem attacks were out of line on this site. I see nothing but personal attack in Ms. Patton's letter.

Joseph Duemer , Professor at Clarkson University, at 10:31 am EDT on March 12, 2007

Mr. Scott, These are undergraduate institutions. Students can pretty much change majors at will. So, institutional ranking is probably helpful.

Whether or not ranking is a good thing (I happen to think it is) is another question. My belief is that ranking allows, to some minimal extent, students to find out more about schools than they would not otherwise know. A high school student (that isn't the child of elite parents) has little way of telling one school from the other, and little way of knowing what its reputation amongst people that matter is.

Mr. McElroy, I am curious, what factors WOULD you like USN to rank? Maybe this is where we can start the conversation.

Larry , at 10:31 am EDT on March 12, 2007

US News & World Report is a commercial enterprise that runs its methodologically dubious but highly profitable college rankings scam in the guise of journalism. Almost all colleges and universities collaborate in this shoddy system because “everyone else does” and nobody wants to suffer any ranking decline by taking a principled stand against the ranksters. (Why does this remind me of a protection racket??) Parents and students should wonder why the intellectuals who run these schools engage in such an intellectually dishonest activity. And they should wonder about the quality of the leaders in America's most admired colleges and universities, who well know how bankrupt the rankings are, yet are silent about them and seem unwilling to develop a viable alternative.

Mommy , at 10:55 am EDT on March 12, 2007

There are alternative rankings of colleges and graduate programs, but they are not as popular for two reasons: 1) they are generally not commercial and not as aggressively marketed as USN; and 2) USN's popularity and “credibility” is sort of circular – whether or not people agree with their methodology, they think other people do. But, consider this: no matter what ranking system is used, colleges will try to game it. Library size will be inflated, class ratios will be manipulated, grades will be inflated, numbers of professors will be fudged, etc. etc.

As a result of this sick system, most highschoolers can't go too wrong in relying on USN, simply because other people do.

LArry , at 12:05 pm EDT on March 12, 2007

SAT and US News

As long as US News discloses its methods for estimating missing data, it is neither “fraudulent” nor “made up,” it is estimated. People who think SAT testing is so limiting, I fear, protest too much. Those who don't report SAT scores are most likely those who score poorly. The SAT does a good job of measuring basic vocabulary, math, and writing skills. I think that testing and ranking are both very useful to parents like me. I wouldn't want my kid to go to a college that was scared to report the average SAT scores of applicants. What exactly are they trying to hide? If they want 200,000 of my hard earned dollars, they had better not hide behind this PC garbage of SATs being “optional.” If the kids can write so well, they will surely do exceptionally well on their “new” SAT with the compostition portion.

teaching mom , at 12:05 pm EDT on March 12, 2007

Reputation score

Obviously there's a big market for rankings, so they'll no doubt continue, whatever the degree of cooperation from institutions being ranked.

However the reputation score in the USNews rankings is indeed a great problem. As 25% of an institution's final score, it essentially serves as a brake to slow any rapid change in an institution's overall rank.

After all, how could presidents and deans, the ones who submit these scores, have a updated and accurate idea about the many institutions they are asked to rank? Don't you think they mainly go by what rankings these same institutions have had in the past?

It's more than legitimate to ask why USNews insists upon giving such a high coefficient to this part of their system, and for institutions to refuse to participate in it.

Jack , at 12:06 pm EDT on March 12, 2007

Alterantives to rankings are...... what?

I'm a student and want to pick the best school for my ONLY four years as an undergraduate or I'm a parent and I want to make sure my $100,000 is invested wisely.

Where do I look? What do I look at? Should I be wary of schools who don't want to be evaluated?

Its a shame that many schools appear to be reluctant to be evaluated. The alternative to the type of numbers USN&WR are desks at college fairs, glossey mailings and word or mouth from ‘oh, your cousin went there...'.

An alternative are high school counselors... many of whom seen to promote small schools who are having trouble filling seats. hmmmm.

WeiBing , at 12:30 pm EDT on March 12, 2007

I think that the US News rankings are useful but somewhat bogus. Every thesis writer at some Ivy league schools, for example, is considered a class of one student. SAT optional schools do get a 30-60 point pick up in the score category. Alumni giving rates are suspect. Besides the peer ranking, data manipulation, is rampant among the colleges. Students like the ranking, however, so let's stop whining unless we want national standards or a national test to be the way admissions is determined. That is the way it is in the rest of the world.

Jane S. , at 12:45 pm EDT on March 12, 2007

U.S. News vs. Colleges

U.S. News should not make up data (even if they flag it as made up). What they've done is threaten to publish false data if the college doesn't play ball. It is easy enough either to put N/A if data is not provided, or to flag real data if it is from a college that does not require the SAT.

Having said that, real data is so hard to find that U.S. News, and, most importantly, its ability to pry standard data out of colleges, serves an important function for potential students. If such data were freely available to students, they would not need to rely so heavily on U.S. News and their infamous rankings to acquire information.

Case in point: Consider the colleges in the SREB (Southern Regional Education Board),a large group of state universities. Recently, Auburn conducted an internal study on its status within the SREB based on important data indicators. You'd think they'd be able to acquire information from their fellow institutions. You'd be wrong — what they did was get all their information from the U.S. News annual issue!

There are two sides to this coin, both of them tarnished.

Bob at State U. , at 1:00 pm EDT on March 12, 2007

The Wrong Solution

Institutions of higher learning can find fault with much in the US News ranking system; so can parents and students. But holding one's breath to gain attention is hardly the answer.

There is a legitimate need for some mechanism to evaluate schools otherwise the US News franchise wouldn't be so widely used. But instead of simply refusing to participate (which is what some of these institutions seem to prefer), why not assist in creating a more legitimate ranking system?

Just as the SATs aren't the only means schools use to evaluate students, the current ranking system isn't the only criterion students and their parents use in evaluating schools. But they both provide information of value for their users. Why not improve the current ranking system rather than try to wreck it?

rob , at 1:00 pm EDT on March 12, 2007

Use News Profit Motive to Advantage

The article and comments note serious dissatisfaction with the present method of college rankings.

US News would like nothing better than to be the leader in the ranking methods. It would behoove educators to use the News desire to be relevant and its ability to reach the public by telling them what would be valuable information.

Presumptions of good education enjoyed by the elite would be diminished by real criteria that measured the difference in ability between the incoming students and the graduates.

The measure of success should be on the quality of teaching demonstrated by the opportunity afforded upon graduation and the five or ten year history of performance of the educated student.

The outcome desired would also be relevant. For example, want to go to med school, try Westminster in Fulton, Mo. Want to be an ad agency art director, try the Brooklyn Art School. Get the drift.

Quizzical , at 1:30 pm EDT on March 12, 2007

Sarah Lawrence complaints contradictory

One consequence of Sarah Lawrence's complaints about U.S. News and World Report's plans regarding SAT score measurement is to reveal that the institution may have eliminated SATs from their recruitment and admissions process, but they are still concerned about them to some extent as far as their reputation is concerned.

It's one thing to complain about an inaccurate measurement, if you held the measurement in good standing at all, but as Sarah Lawrence appears to hold something between utter disregard and total contempt for the SAT, it is difficult to see why any SAT measurement delivered by U.S. News and World Report, no matter how flawed the estimate, would be of consequence to the school.

If you are committed to the idea that the SAT is a flawed instrument, then the scores and U.S. News's estimates should be meaningless. If they are not meaningless, then obviously those scores have some value and value beyond only the marketing/recruitment value. Sarah Lawrence would be better served by either moving to SAT-optional status or using the U.S. News rankings to further demonstrate their committment to more comprehensive forms of assessment, by ignoring the magazine's decision and pointing out the unimportance of it in the school's own marketing initiatives.

Clint Brooks , at 2:25 pm EDT on March 12, 2007

Kudos for Myers (finally)

Those lambasting Sarah Lawrence for making it more difficult for parents and students to make an informed decision clearly don't understand how to make an informed decision in the first place. Among other things, the point Myers and others are making is that “rankings” based on only test scores are a misguided and uninformative approach.

Take the case of Sarah Lawrence — the school doesn't supply its students with grades unless they specifically request them. It doesn't even give them tests. Of what use, then, is a test itself (the SAT) as an indicator of how well the student will do in college? Instead, Sarah Lawrence requires papers — a lot of them — as measures of a student's progress. Hence, it also requires students' applications to contain lots of writing. To do otherwise would be absurd. (To put it another way, would you interview an applicant for a sous chef position by giving them a mechanical engineering test? No, you would give them a cooking test.)

Would this approach work for all schools? No. Because not all schools require students to write. Many require tests, and for those institution the SAT will continue to remain a valuable assessment.

The problem with US News, then, is that it uses the exact same criteria to rank completely, wildly different schools. Ranking Sarah Lawrence on test scores is senseless and it poisons the rankings. It makes them meaningless. THIS is what Myers and others are protesting — not the fact that there is an organization attempting rankings, but that it is doing them incorrectly.

Alexis Turner , Sarah Lawrence, ‘99, at 2:30 pm EDT on March 12, 2007

SLC does, indeed, test

First of all, using plug numbers and assumptions is not “fake” data, so long as people are honest about it.

Alexis,

For what it is worth, I found a few examples of things that sure look like tests on SLC's website:

In a Calculus course, at http://pages.slc.edu/~dking/Statistics_Syllabus.doc a series of “problem sets” is given (“Work on problem sets is do be completed independently.”)

Likewise, in “Economics: Mainstream and Radical Perspectives” the “Five highest quiz scores = 20%” http://pages.slc.edu/~froosevelt/semester_syl.htm

And General Chemistry has lots of quizzes. http://pages.slc.edu/~rhinrichs/general/index.html ! 20 of them!

Larry , at 2:55 pm EDT on March 12, 2007

I actually fill out much of this survey...

I personally fill out much of the U.S. News survey for my institution. It's painfully extensive, and I dread the annual arrival of the U.S. News survey every March.

Much has been said. I'll mostly limit my comments to what hasn't been said.

What about classification? The Carnegie single classification system was revised in 2005 and “was replaced by a set of multiple, parallel classifications” (carnegiefoundation.org). But U.S. News still uses the old Carnegie single classification. The result is that some colleges are classified as liberal arts and nearly identical undergrad-focused colleges are (mis)classified as master's institutions due to historical categorizations. That's a huge disservice to students/families using the rankings to develop a prospective college list, and to those colleges that don't fit in the old (but unchangeable) classification they are placed in by U.S. News.

Rankings do a disservice to the idea of finding a good “fit” for college. On the flip side some of the data reported might help in that effort for students/families that don't get caught up in the rankings themselves. U.S. News, due to it's ubiquity, is a convenient way for parents and students to discover this data.

One positive of these rankings is that it puts (some) colleges that might otherwise be overlooked on the radar of prospective students. My subjective evaluation based on conversations is that it benefits top tier regional institutions the most. (And if you think my subjective evaluation is dubious, it's no more ill-informed than the college president reputation ranking.)

On that note, I second that peer reputation information is dubious, and having it count at all is questionable, much less 25% of the overall score. College presidents guess at reputation, and reputation (positively and negatively) lags reality at any given institution. I'd like to see college presidents boycott this section of the survey. As U.S. News really has no other means of obtaining this information, institutions could take a stand on this issue.

That's all for now — busy working on the myriad other surveys sent to colleges...

Survey Respondent , at 3:25 pm EDT on March 12, 2007

U.S. News

As an administrator in higher education, I have experienced the intense pressure that accompanies having to maintain a current ranking or, better still, achieve a higher ranking in U.S. News. I have learned that responses to the questions sometimes have less to do with presenting a clear picture of exactly what educational outcomes a particular institution has to offer in comparison to other institutions, than with the ability to present the data in just the right way to paint the picture that will impress those seeking the information. Administrators who are particularly skilled at understanding how to respond are worth their weight in gold. The drive to manage the data can be observed through the process of sending questionnaires to college presidents and vice presidents. These documents are mailed once or twice a year by U.S. News, asking the officers what they “know” about other collegs and universities. These mailings are always preceded by mass mailings from most of the institutions on the query list,to all of other institutions on the list. It is not unusual for colleges to receive over 100 brochures in any one year. Does anyone actually look at these brochures before or at the time they are filling out the reputational questionnaire? The more important question is, for those attempting to compare and contrast institutions of higher education, how do these processes impact on the non-academic's ability to make right decision for themselves or their children. Perhaps there is a way to keep the more objective, quality based information offered by U.S. News while eliminating some of the less reliable, more easily massaged data. Dr. Donne Kampel

Donne Kampel , at 3:25 pm EDT on March 12, 2007

As yet another former Sarah Lawrence Student (class of ‘05) I must confirm that there are tests given. However, if you will note the examples given above they are all reated to what might be considered “Hard Sciences and Mathematics". Frankly, testing at Sarah Lawrence is restricted to only a very few specific types of classes (mnath, science, languages) and the vast majority of students at Sarah Lawrence never, or ver rarely, encounter a test of any kind. Indeed, there are no majors at Sarah Lawrence, and very few overt requirements. One need never, ever, take a math or science course in their time at SLC or one may take primarily math and science, it is the students choice. Whether you agree with this method the fact of the matter is that education at Sarah Lawrence is vastly different from that of most other undergraduate institutions. To compare them is, to use an old cliche, like comparing apples and oranges. Both may taste good but they are not the same.

Daniel , To Test or not to Test, at 3:35 pm EDT on March 12, 2007

“Those lambasting Sarah Lawrence for making it more difficult for parents and students to make an informed decision clearly don't understand how to make an informed decision in the first place.”

Duh.

Shouldn't we ‘professionals' make it easier for parents and students? Or should we replace the SAT with the task of simply figuring out what a school requires?

Weibing , at 3:35 pm EDT on March 12, 2007

Susan Patton — Fourth Tier?

Susan Patton shouldn't be calling Sarah Lawrence a “third-tier” school (which it surely isn't and which has been nicely ranked previously by US News) when she writes “. . .would risk it's reputation. . . .”

Ahhh, it's a shame, Ms Patton.

Karen Lane , at 3:45 pm EDT on March 12, 2007

SAT optional and retention

In response to the parenthetical remark “(Morse's thinking is based, of course, on the assumption that the students who don't report SAT scores will have lower retention and graduation rates — something that colleges that have gone SAT-optional say is not the case.)", I can confirm that so far — with complete data on 2 entering classes at Pitzer College, a member of the Claremont Colleges — our SAT-optional policy does not indicate any lower retention for students not reporting SAT scores. In fact, students submitting SAT scores have a slightly higher attrition rate. Other data analysis suggests that HS GPA is a statistically better predictor of college GPA and graduation rates than SAT scores.

Peter Nardi , Professor of Sociology/Director of Institutional Research at Pitzer College, at 4:45 pm EDT on March 12, 2007

Let me say this. As a current SLC student who has enjoyed his time there less than most and could be considered an authority on the flaws of the school, I firmly stand by President Myers decision to stand up to both US News and teh College Board. Ms. Patton is grossly uninformed about SLC disturbingly adimant about her representation of false information (SLC is in truth, a top Liberal Arts school). Although I have my issues with SLC, it is about time that an educational institution stand up to organizations whom they have catered to for far too long.

Jack , at 6:25 pm EDT on March 12, 2007

Why rank at all? If U.S. News, and the other supporters of rankings are really concerned with transparency, why not just report the data and let consumers decide what is most important? The rankings actually obscure the data behind one number, letting readers stop as soon as they see where the institution falls in the hierarchy.

Dan Laitsch , Why rank at all?, at 6:25 pm EDT on March 12, 2007

Response From U.S. News & World Report

U.S. News released the following press statement and requested that it be posted here:—Inside Higher Ed

In a March 11th op-ed in The Washington Post, the president of Sarah Lawrence College, Michele Tolela Myers, expressed concern about how the college's decision to eliminate SAT scores from its application process would affect the college's ranking in upcoming editions of America's Best Colleges, published annually by U.S.News & World Report. The op-ed stated that U.S. News had decided on an approach to this situation, when in reality, we have not yet done so.

Sarah Lawrence's decision is unique, and the magazine's handling of it is still under consideration. Some colleges have made SAT or ACT scores optional in the admissions process, but to our knowledge, no other major college has decided to disregard them completely.

Our rankings are painstakingly tabulated, using the best data available. U.S. News data researchers regularly participate in briefings and conferences where the most complicated nuances of the process are discussed with the ranked institutions. We regularly adjust to changes in the educational environment, and we plan to address this circumstance in a similar manner.

US News , at 7:31 pm EDT on March 12, 2007

Dan's right...

While one student may be wholly concerned about reputation, another might be more interested in the likelihood of seeing a full-time professor, with an office to hold office hours in, in a class in the first two or three years of a degree.

Rather than helping students and their families make informed decisions, _US News_ dumbs down the process, behaving as if only a handful of factors matter, with college presidents' opinions of one another (and each other's schools) among the factors of foremost significance.

I would suggest full funding for the National Center for Educational Statistics (a few years ago, it eliminated some statistics it had long collected in response to budget cuts) and a requirement that a link to its rankings and measurements be required on college application pages as part of full disclosure.If colleges want to compete for the best applicants, providing meaningful information that the diligent can sort through to make truly informed decisions should be part of that competition.

It's not that statistics lie so mucha as that they invite misuse. For example, if 20% of your students participate in a cooperative education program in which semesters in the field alternate with semesters at the school, a number that merely expresses that it takes students, on average, longer to graduate from your school hardly does your school justice. And averaging that number in with a bunch of other numbers of varying degrees of meaningfulness makes it pretty likely that a program that many people would see as a strength will simply make your school appear weaker in the _US News_ sweepstakes.

Thane Doss , at 10:20 pm EDT on March 12, 2007

Nothing new...

I remember I met a graduate student at a conference, at night over drinks, once who told me that he was instructed to make up numbers to get his institution's rankings to rise. When we asked how he felt about cheating the system, his response was “What, are the U.S. News and World Report police going to come and arrest me?” When will colleges band together and recognize how ridiculous these rankings are? When will U.S. News get some actual integrity and end this silliness?

PS , at 4:35 am EDT on March 13, 2007

US News — Useless

New book out with the shortest chapter in literary history. The chapter is the author's view of the rankings of US News. It's one word:

Useless

Book's on Amazon: How To Win The College Game, by Paul Lloyd Hemphill

Payul , at 8:01 am EDT on March 13, 2007

SAT

I recently visited a public university in south NJ with my son. After the tour we went to admissions. We were told the selection to the school was ONLY based on SAT and GPA and the fact that he is an Eagle Scout and very active in the community DID NOT MATTER AT ALL. He has a 3.7 GPA and average SAT scores. I taught him to help people and be involved. I guess I was wrong. The fact is he should have done nothing but work on SAT scores or continue with one sport and get a free ride like some of his friends that are not as smart.

TOM , at 10:46 am EDT on March 13, 2007

One parent's ranking

I must respond to Susan A. Patton's description of Sarah Lawrence as a “third-tier” school. My three children, all excellent students, graduated from three different colleges: Harvard, Sarah Lawrence, and Brown. All received a fine education. I think rankings are absurd, but if forced to rank these three colleges, based on my close observations and experience, I would put Sarah Lawrence far above the other two.

Ed Miller , at 11:00 am EDT on March 13, 2007

Tom,

I had a similiar experince with a public school in Michigan. Their comment was something like “everyone works at church and is a cheerleader..”

That statement was a big help to me though in deciding that this was not the school for my daughter.

stm60 , UConn, at 11:00 am EDT on March 13, 2007

SATs and SLC

I am a graduate of SLC. In ‘96, when I applied to colleges, SLC was a 1st tier liberal arts school according to US News. My SAT score was 1350, which was a little bit higher than the average score at SLC at that time. Their ranking was important to me but it wasn't the deciding factor. I'm there now as a graduate student and as far as I can tell it continues to have an incredibly rigorous undergraduate program. I'm sure SLC kicked out there SAT because they don't believe in it, not because their applicants started mysteriously having bad scores. My friends thought that it must be an easy school because there are no test and no grades (unless you ask for them, and when you do be prepared to be surprised how harshly they grade). It made me laugh...an undergrad writes upwards of 50 pages a semester per class and has one-on-one meetings with their professors. They can tell in a minute if you haven't understood the class work or if your thesis is thin. There's no slouching in the back of the room and memorizing a text book at the last minute to pass a midterm. You really have to know the subject matter backwards and forwards in order to succeed.

Heather , at 11:25 am EDT on March 13, 2007

From Both Sides

As an SLC grad and former reporter for one of Mort Zuckerman's outlets (not U.S. News), I can see both sides of the issue here. Though, I have to agree with President Myers. I watched as U.S. News continually knocked down SLC's ranking since I graduated. This coincided with the school building new facilities, expanding the curriculum, and building upon and maintaining their core faculty. Their was an obvious disconnect between U.S. News findings and the quality of an education being offered at Sarah Lawrence.

U.S. News is carrying an extraordinarily large stick in coaxing colleges and university to cooperate with their research. When a college — like SLC — criticizes that reporting process it only serves to add some transparency to U.S. News' findings. Sarah Lawrence is an iconoclastic school and when Myers broadcasts that in a national publication, I'd speculate that compensates for some of the PR damage caused by U.S. News results.

An instructive note about U.S. News rankings: Before going to Columbia Journalism School, I referred to U.S. News for information about other grad programs to find out that they do not rank journalism programs at the graduate or undergraduate level. The reason, as I understood it, was a conflict of interest. That didn't pass my smell test even then. The more likely reason: U.S. News editors didn't want the scrutiny of hundreds of reporters and editors peeved at how their alma mater's were ranked.

A Former Student & Reporter , at 1:20 pm EDT on March 13, 2007

The Pedagogy

I attended a Catholic Institution very highly ranked by US News and World Reprt and now I'm a graduate student at Sarah Lawrence. SLC is a rigorous and challenging institution that requires very bright and very motivated students. Heather is correct. Students here need know their topic exceedingly well, be able to form and defend their own opinions and love the responsibility to learn and grow intellectually. The Catholic University that gave me my BA valued NONE of these abilities and I left that institution unchallenged. Sarah Lawrence, on the other hand, takes great pride in it's students as individuals, it values differing opinions and it is the finest community I have ever known.

Sheila , graduate student at Sarah Lawrence College, at 1:35 pm EDT on March 13, 2007

I'm a newspaper reporter for a largish mainstream newspaper. Every university on my beat trumpets their rankings in press releases and I get constant pressure from my editors to write stories about the rankings. I am loathe to do this after I actually called the reporter a number of years ago who was charged with creating the rankings, only to find that they were publishing two-year-old rankings as if they were new, and using some of what I considered to be very shoddy and questionable practices in other ways too. Every year I have to fight with my editors to NOT write about the rankings even though I totally believe in my heart they are questionable at best! If every college ignored their surveys, there would be nothing to complain about!

Newspaper reporter , at 4:05 pm EDT on March 13, 2007

The Survey has arrived

The U.S. News Survey login information is arriving at colleges across the country today. Let the games begin...

In response to the reporter in the previous post, even if colleges refused to fill out the U.S. News survey, U.S. News college ranking would continue. They'd simply pull much of the data from other sources, such as IPEDS. This ranking is too much of a money maker for them to stop publishing it, even if there were no direct survey respondents.

And as the reporter mentioned, colleges are complicit. They publicly trumpet favorable U.S. News rankings as a sort-of “Good Housekeeping Seal of Approval,” even when they privately doubt the validity of the rankings. For some colleges, it's the bedrock of marketing campaigns to prospective students.

Survey Respondent , at 4:16 pm EDT on March 14, 2007

There IS A Solution To This Problem!

Sorry to be weighing in so late, but I have a solution to this problem ... and it IS a problem. It's not a problem to academics who should (but act like they don't) know better, but it is to parents and prospective students, many of whom really don't know any better (and are not helped by USN&WR, the complicit colleges and universities, or the media).

The solution is NOT to refuse to participate ... unethical as they are in pursuit of this fast buck, USN&WR can work around that. The solution is to give them nonsense information. When the office of the president of Harvard reports to USN&WR that 37% of Harvard's faculty have terminal degrees, the Crimson's student faculty ratio is 53 to 1, and she (the president) would rank Yale number 146 amongst American universities – and when everyone else does the same – well, won't that be fun? And will this hurt Harvard's quest to attract their “optimal” student body? Give me a break!

And wouldn't we all enjoy watching the gyrations over there at USN&WR as they dream up ways to lie through their teeth to salvage this cash cow?

Frizbane Manley , at 9:55 am EDT on March 16, 2007