The paper was rejected, which was fine. I was a second- or third-year graduate student, it was an early paper, and rejection was the expected outcome. What wasn’t fine was the report. Alongside the mathematics it had an opinion about me: that the junior person needed to learn how to write a paper. I took it to a senior colleague, and before they said anything about the report they asked a question I’ve thought about since. Was I upset about the outcome, or about the message?
I had an answer. The outcome I could live with. The message had been handled badly, by someone who couldn’t be seen and knew it. What I didn’t know then was that the question was the whole thing. It sorted the report into two piles, only one of the piles was about the paper, and most of the sting was in the other one.
Bad is stronger than good
Psychologists have a name for the general pattern, and two reviews organized it. One is titled, plainly, “Bad is stronger than good.” The other broke the pattern into parts. A negative event outweighs a positive event of the same size. A bad thing looms larger as it approaches. Mix a good thing with a bad one and the whole reads worse than the sum: a strong talk with one wrong slide is remembered as the talk with the wrong slide. And bad events get more attention and more words; languages have more vocabulary for what hurts than for what pleases.
The usual explanation is that the two kinds of error cost different amounts. Missing a threat can kill you. Missing an opportunity usually doesn’t. A system tuned that way is right in expectation, and we inherited the tuning.
Two cautions. The magic ratios sometimes attached to this, five good interactions for every bad one, three positive emotions for every negative, don’t survive scrutiny; one of them turned out to be mathematical nonsense. And the bias isn’t the whole story. Given a weak or neutral input, people lean slightly positive; it’s at higher intensity that bad takes over. On an ordinary afternoon of computing, the baseline is mildly good. It’s the evaluated moments, the report, the panel, the decision, where the asymmetry runs.
One harsh report
One harsh referee report against five good ones. Everyone knows which one you’ll read twice. But three things are happening at once, and only one of them is the bias.
The harsh report carries more information. A positive report says accept. A negative one is three pages with line numbers. In Sounding Sure I argued that failure is the high-information event in a research career, and this is that: the referee who found something wrong has told you something, and the referee who didn’t has mostly told you they didn’t look. Attending more to the harsh one is correct.
The harsh report also carries more feeling, and the feeling scales past the information. The derivative is right and the integral is wrong. It’s right to weight the negative report more in deciding what to change. It’s wrong to let it set the level of how you feel about the paper, or about yourself.
And the sample is skewed before you touch it. The most common feedback in this profession is silence, and the breaks in the silence lean negative, because an error generates email and a correct paper generates nothing. The non-silent feedback you receive over a career was drawn from the negative tail by the field, before any bias of your own got near it.
The weakest step
Mathematics is unusual ground here, because in our content the bias is correct. A proof with one wrong step is not a proof. As an argument it is exactly as strong as its weakest link, however good the other forty pages are, and the other forty pages are often good: a lemma that holds on its own, an idea that works in a different setting. The theorem is void; the paper isn’t. But at the level of the theorem, negativity dominance is the logic of the subject. Our training is a trained search for the flaw, and refereeing is a job whose useful output is mostly the negative findings.
The trouble is where that logic goes next.
Which information counts, when you’re judging a person, depends on what you’re judging. For honesty, bad behavior is the diagnostic kind: an honest person doesn’t steal once. For ability, it’s the other way. Anyone can fail at a hard thing; only an able person succeeds at it. One correct proof of a hard theorem should tell you more about a mathematician than one failed attempt.
That isn’t how the field reads it. A wrong lemma gets filed under carelessness, a question about trust, rather than under ability, a miss. I think the reason is attributional. In an empirical science a wrong hypothesis was refuted by the world; the failure sits outside you, the data didn’t cooperate. In mathematics there is no outside. The error is in an argument you produced, and the check that would have caught it was available to you the whole time. Internal and controllable is the configuration that draws blame rather than sympathy. A failed experiment is unlucky. A wrong lemma is careless.
The field does honor some failures. A wrong conjecture can be a career’s best work, and a failed approach at the frontier gets celebrated: Kummer’s attack on Fermat gave us ideal numbers. What doesn’t get forgiven is a checkable error left unchecked. And the forgiveness doesn’t extend to the ideas sitting next to the error. The error contaminates them, and the attempt is written off whole. I’m not sure this exists to the same degree in other disciplines. From outside them, the goal looks like getting the answer, and the wrong answers help get there and are valued for it, even if valued less. Ours is the field that treats a wrong step as a finding about the person who took it.
Two columns
Here is how it actually sorts, at least for me.
An honest negative report I take as fair. It’s part of the process, it makes the result better, and I don’t lose much over it. The unfair ones are different. They read like an ax being ground, or someone taking things out on whoever happened to land in their inbox. I feel those in the moment. They don’t last, and the extreme ones I mostly remember as funny, including the one from my second year.
For a long time I thought that was temperament. I now think it’s a sort, and it runs on the same two columns the field uses, pointed the other way. Research on fairness finds that people accept bad outcomes from processes they judge fair; what wounds is the sense that someone acted badly. The fair report goes in the ability column, where a negative is information and you act on it. The unfair one goes in the morality column, where a negative is diagnostic, and what it diagnoses is the referee. The senior colleague’s question was asking which column the report was in. For most of the sting, the answer was the second.
Which raises the question of where the unfair reports come from. My observation, over enough years of them, is that the harshest tend to come from people somewhat insecure in their own place, and that invisibility lets them be critical of the person rather than the work. Both halves have support. People whose own standing has just been threatened judge others more harshly, and letting them shore up their self-image first removes the effect. Anonymity doesn’t create the impulse; it removes the price. The modest evidence from journals that tried signed reviews points the same way: signed reports came back more courteous, with no loss of quality.
If that’s right, an unfair report is information about the referee’s position and very little about the paper, which is how I’d been filing them anyway. The field’s failure is that nobody tells a second-year graduate student this is a live possibility before the first one arrives.
What it does downstream
It thins the record. If a checkable error reads as a character finding, people stop showing attempts. Partial results stay in drawers, failed approaches never get written up, and the field loses its account of what was tried. The arXiv shows the theorem and never the four approaches that didn’t work, so everyone at the frontier rediscovers the same dead ends alone. In Behind at the Beginning I noted that the field publishes outcomes and hides process. This is one of the mechanisms doing the hiding, and it’s strongest exactly when the drawer is most expensive to open: on the job market, a visible error costs more than an unpublished attempt, so the attempts stay unpublished.
It reproduces the referee. Where does the insecurity come from, in the person writing the harsh report? Partly from reports like the one I got. The culture that produces harsh refereeing is produced by harsh refereeing, with anonymity removing the price at every hop. It’s the same transmission I described in I Only Looked at This Last Night, where a sentence costs the speaker nothing and the room believes it. My early referee was, very likely, somebody’s author once.
It multiplies by stage. The bias doubles every negative. Concentration multiplies it again. I have a practice I’d recommend to anyone: once a paper is sent, I’m done with it. I move to something else until the report returns, or a colleague’s comments appear, and then the paper gets updated based on the information. This works on the dread, which is the bias in the time domain. A bad outcome you’re waiting for is felt over the whole wait, while the event itself is a point; people in experiments will take an electric shock sooner, and sometimes a larger one, rather than sit through the anticipation. You can’t shorten the wait. You can deny it attention by having somewhere else to be.
But that requires somewhere else to be. One pending paper among six projects is one draw, and I can look away from it. A fourth-year student has one paper, and it’s the thesis. A postdoc on a two-year clock can have a season riding on a single report. The people most exposed to one negative outcome are the ones with the least capacity to look away from it. Send it and leave is correct advice, and it’s also a sentence written from the safe side of the table.
And the losses come doubled. Criticism isn’t the only thing that hits harder. Not getting funded is a loss of standing, and it’s also the sting of not having the resources to travel or support students. When the administration cut lines from our graduate program, we lost some standing and we lost part of the thing that was important, working with junior mathematicians. Those are two losses from one event, on two ledgers. The standing loss is a circumstance, and circumstances fade. The other one compounds: students not supported, the next proposal written from a weaker position, which psychologists who study resource loss call a loss spiral. And in Go Celebrate I argued that it’s the activities, not the landings, that adaptation doesn’t cover. Those lines were students, which is to say an activity.
What to do with it
Four things, in increasing cost.
Sort before you feel. Before responding to a report, put every comment in one of three columns: about the work, about the process, about the referee. Only the first column goes into the revision. The second goes to the editor, if it goes anywhere. The third is information about a stranger. Do the sort before the feeling sets the level, and keep running it afterward, because once you know the bias exists, that’s just my negativity bias becomes a way to dismiss a fair report, and the columns are the defense.
Send it and leave. With the caveat above. If you have only one thing, the next best move is to build a second, however small, so the wait has somewhere to go.
Write every anonymous report as if it were signed. Criticize the work and not the person. I try to treat people differently than I was treated. I try to be fair to the process, and to step aside when I think I might be biased. I probably fail at this in some ways, but recalling that there is a person on the other side of the report helps, and the bias means every sentence I type will be weighed at twice its size on arrival. I’ll never see what a report cost. The only check is the one I run before sending it.
If you have standing, separate the error from the person out loud. When a student brings a wrong lemma, treat it as a miss, and say what was good in the approach before saying what broke. In letters and in seminars, let a failed attempt be an attempt. If you edit, send the third column back to the referee instead of forwarding it to the author. A senior person’s criticism arrives at double weight and the sender never sees it land, so the correction has to be built in before sending, not after.
I still ask the senior colleague’s question, now of my students, when a report comes in that has an opinion about them. Outcome or message? They usually have an answer, and it usually sorts the way mine did. The outcome they can live with. The message was in the other column.
The report from my second year was wrong about me, I think, though I did learn how to write a paper eventually, mostly from editors who criticized the writing and not the writer. What it was right about, without meaning to be, is that there’s a person on each side of a referee report, and only one of them can see the other.