Читать книгу The State of Science - Marc Zimmer - Страница 28

Part Two

Doing Science

Chapter 4

The Nuts and Bolts

This chapter describes how scientists know whether they can trust published results, how the peer-review system works, and where scientists get funding for their research.

Peer-Reviewed Publications

In 2018, Susan Bourne, the interim dean of science at the University of Cape Town in South Africa, and I were talking about science journals. The scientific paper is the primary mechanism for both broadcasting one’s own scientific results and determining what research has been done by others and how they did it. It is also a measurement used to assess a scientist’s worth, whether for funding, tenure, promotion, or getting a job. I like how Susan succinctly summed up her thoughts on the scientific publishing process and probably those of most other scientists: “The system is completely crazy. The taxpayer funds the research, the scientists do the research, write it up for free, do all the editing, all the peer reviewing—the publisher gets everything for free. And then the taxpayer has to pay for us to get access to it again.”

Prior to the 1600s, scientists privately communicated their findings and ideas in letters, gave public lectures, and wrote books once the experiments were all done and their ideas and theories had matured. There was no way of publishing increments of one’s research. When the advent of scientific journals allowed scientists to publish chunks of their work, “scientists from that point forward became like the social insects: They made their progress steadily, as a buzzing mass.”^[1] At first the papers were shorter, less formal, and more readable than they are today. As the research became more specialized, papers became longer and contained more jargon.

I sometimes think of scientific papers as puzzle pieces. Nature is full of intriguing puzzles for researchers to solve. The jigsaw pieces don’t come in a box, with the number of pieces listed and a picture of the solution on the lid. To solve any of nature’s puzzles, researchers need to find the pieces, then try to put them in the correct place. Some puzzles are much more important than others, and within the puzzles themselves some jigsaw pieces are more central than others. Scientific research is all about finding the pieces, putting them together, and trying to extrapolate to determine the big picture even when some pieces are still missing. Some puzzles lead to new understandings, others form the basis of new theories, and yet others result in new techniques. When a puzzle reaches a certain stage, it becomes easier and easier to put in the pieces; the research accelerates. The breakthrough occurs when the pictures on the puzzle become visible, when a central piece is placed that allows whole new areas to emerge. An important puzzle can lead to the start of many other new puzzles.^[2]

Each year about 1.8 million papers are published in roughly 2,800 journals.^[3] The journals are not all equal. Science, Nature, and Cell are the most prestigious ones; the important puzzle pieces are published in them. Getting a paper published in one of these journals assures the authors of a wide readership and significant prestige, and in turn the reader knows that the papers have withstood rigorous peer review and have been judged to be of importance to all scientists. The “impact factor” of a journal (the annual average number of citations per paper published in the journal in the previous two years) is an attempt to quantify its prestige. (The 2018 impact factor for Science was 41.1, which means that the average paper published in Science in 2015 or 2016 was mentioned in 41.1 papers in 2017. The parallel impact factor for Nature was 41.6, almost the same.) Publishing one’s work in the journal with the highest impact factor is important and something of an art. Being too ambitious in journal shopping leads to rejections and delays, while taking the safer route and submitting to a journal with a lower impact can lead to less of the needed exposure and prestige that parlay into grants, jobs, tenure, and fellowships.

Nature receives about 200 manuscripts a week but can publish no more than 8 percent of them. Upon receiving a manuscript, a staff editor with expertise in the area covered by the paper makes a first cut and within a week decides whether the paper should be sent for external review or be returned to the authors.^[4]

In the next step, Science and Nature, like most other science journals, use a single-blind peer-review system to evaluate their manuscripts. The papers are sent to at least two external referees, who also have expertise in the research area covered in the paper. In the single-blind process the reviewers know who wrote the paper, but the authors never officially find out the identity of the outside experts (although journals have begun experimenting with giving reviewers the option of signing off on their reviews). Science, despite its huge expanse of subjects, can still lead to remarkably small circles of experts, especially given how highly specialized and specific certain subjects can be. As a result, it’s often not out of the question that researchers could accurately guess who their reviewers are. This single-blind process creates some problems of bias against women (see chapter 2) and in favor of well-known researchers from prestigious academic institutions.^[5]

Peer review is not new; it has been around for at least 350 years. Henry Oldenburg (1618–1677), the editor of The Philosophical Transactions of the Royal Society, may have been the first editor to use the system. Since the 1960s the number of journals has ballooned, and the need for impartial experts capable of reviewing scientific manuscripts has grown. Peer review is done on a voluntary basis; academics are not compensated for the reviews they write. The majority of scientists see this process as pivotal to scientific progress, and as such they are willing to volunteer their time to peer review. That doesn’t mean, however, that they are all happy with the process as it stands. More than 200,000 academics track and verify their peer reviews on a website called Publons. In 2018, the service analyzed the million-plus reviews conducted for the 25,000 journals in its database and conducted “The Global State of Peer Review” survey of more than 11,000 researchers.^[6] It found that most editors are from “leading science locations” such as the United States, Europe, and Japan, and that they tend to look for reviewers in their own backyards. Consequently, researchers from the United States, United Kingdom, and so forth write nearly two peer reviews for each manuscript they have submitted, whereas researchers from countries such as China, Brazil, India, and South Africa do about 0.6 review per submission. Reviewers from “emerging economies” are more likely to agree to review a paper. Their reviews are returned more promptly but are shorter. Only 4 percent of journal editors are from “emerging economies.” It takes about five hours to write a review, which means that each year scientists spend about 68.5 million hours reviewing papers without compensation from for-profit publishers. The average review is returned in 16.5 days and is 477 words long. The unpaid cost of peer review in 2008 was estimated to be $3.5 billion.^[7]