Bias Numbers: Less Than Meets the Eye?

By: Greg Mitchell With charges of media bias, left and right, flying faster than ever, a Pew survey on newsroom attitudes released on May 23 was certain to intensify the conflict. Indeed, its section on journalists' political beliefs almost produced hysteria. For some on the right, it wasn't enough that Pew confirmed that self-described liberals, though a minority in the newsroom, far outnumber conservatives. These partisans charged (in numerous articles and in many letters to E&P) that those within the "moderate" majority were actually all liberals in disguise.

Pew, as you may recall, found that at national news outlets, liberals outnumbered conservatives 34% to 7% and at local outlets by 23% to 12%.

Apparently John Gibson of Fox News was referring to the Pew Survey on July 12, when he moved that entire moderate majority to the left, then claimed that "80-some percent of reporters are self-described liberals." Ironically, in that commentary, Gibson was defending Fox News from charges of conservative bias ? even as he illustrated it. The following day, Rush Limbaugh repeated the 80% figure, explicitly citing Pew.

Accurately portrayed or not, the latest Pew results received tremendous attention. And while few could question the general finding that there are more liberals than conservatives at newspapers, is the spread really as wide as the roughly 4-to-1 reported by Pew? A check of the survey's methodology suggests the findings should be taken with a chunk of salt.

When the Pew Survey was released, not many looked behind the numbers to find out exactly who was answering the questions. Few media outlets that covered the results observed, for example, that this survey made no claims of being accurate to X or Y percentage point.

Looking at the methodology provided by Pew online, one finds the entire sample included a total of 547 journalists. It seemed broadly based, however, reaching reporters and executives at national and local outlets, and spanning newspapers, magazines, wire services, TV, radio, and Web.

But was it really a fair sample, at least in the newspaper sphere? Those queried, Pew said, came only from papers in the top 100 by circulation. That's hardly representative, since it left unsampled the vast majority of the nation's 1,456 dailies (not to mention some highly influential weeklies). All of the newspaper journalists worked in cities the size of Syracuse, N.Y., Albuquerque, N.M., or larger.

Then, for some reason, Pew separated out 16 papers (all in the top 25 by circulation) and placed them in the "national" media section, leaving the rest in the "local" category. While some of these papers, such as The Wall Street Journal and USA Today, truly are national papers, can this really be said of the Houston Chronicle, The Arizona Republic, and Detroit Free Press, among others? Yet they were placed in the same category as broadcast TV and cable networks, National Public Radio, and the like.

Another problem: While this "national" sample is geographically diverse, it appears slanted in a liberal direction, at least judged by the newspapers' editorial pages. It included The Washington Post, The Boston Globe, The New York Times, and Los Angeles Times, but not The Washington Times, Boston Herald, New York Post, or Los Angeles Daily News.

Andrew Kohut, director of the Pew Research Center, told E&P that "the listings" used for the May survey "indicated it was a representative sample." He acknowledges, however, that Pew felt it was "stuck with" drawing only on the top 100 papers, and placing the leading papers in the national category, because that's the way it was done in the past ? and Pew wanted to continue doing it this way for comparison. "It is a limitation of the study ... we do not cover journalists working for smaller papers," he said.

Yet you'd never know that, if you didn't check the methodology section.

Dr. Christine Urban, president of Urban & Assoc., which has handled major studies for the American Society of Newspaper Editors, does not condemn the Pew study, but observes, "There are so many small papers, and so much of America is served by those small papers. Any study has to reflect that."

And what about other demographics? Again, they hardly represent the average editor and reporter. According to Pew, the median age for the print respondents was about 50. Better than 1 in 4 claimed a graduate degree.

Frank Newport, editor in chief of The Gallup Poll, explains that the size of a sample is not a "problem per se" if those surveyed are truly representative. Pew, he notes, "generally tries to do things right," and he praises them for listing the newspapers in their methodology. Still, in this volatile age, it would have been nice to get a more credible estimate before the partisans on both sides went to work. Imagine, if you will, a national poll on the Bush-Kerry race, or the war in Iraq, that only queried residents of large cities. Howls of protest would be heard well into the night.