by Pietro Vischia

I have spent the last few hours editing a paper for an analysis I have been working on with some colleagues in the last year, and I am so bored that I stopped for a few minutes to tell you how preparing a paper in a large experimental collaboration works.

The procedure I will be describing is the one that is employed in the CMS collaboration, but other LHC collaborations such as ATLAS, LHCb, and ALICE have similar procedures, differing by minor details. Older experiments worked in a slightly different way, and you can find some details on the inner workings of the CDF experiments in Anomaly!, the nice book by Tommaso Dorigo. By the way, I have put the link of the publisher, but if you want to buy it I would suggest you do so on Amazon. The publisher is horrendously slow in sending the book, and the customer service is horrendously slow in answering your inquiries.

Sorry for the detour, and back to the procedure for preparing a paper!

First you spend a variable amount of time analyzing the data. The process can be extremely fun or excruciating, depending mostly on who you are working for: since typically many institutions want to perform the same analysis, you often have to work with random people that sometimes for various reasons are not pleasant to work with.

All across the process, you keep reporting to the relevant physics analysis group (PAG), which is composed by the people that use to work on analyses on a broad topic (e.g. Top physics, or Higgs physics, or Exotica, etc) and is led by two conveners, that are usually experts on the topic. Normally, since such topics are quite broad, there is a subgroup structure, and you actually report to the subgroup (e.g. Higgs-Exotica, Higgs-WW, Top cross section, Top mass, etc.), and you report to the main group only for the formal steps described below.

You may be asked to perform various checks or add portions of the analysis: when you are done, you are asked to produce an internal note detailing all the aspects of the analysis, and a draft of public documentation: the conveners review the documentation, and if they are happy, you are allowed to present the analysis at a special PAG meeting which is called preapproval meeting.

During the preapproval, the conveners and anyone at the meeting can make questions to check the quality of the analysis. If at the end of the meeting the conveners are happy ,then you are pre-approved, and the ARC review can start.

Four physicists with a doctorate are chosen to be part of an Analysis Review Committee (ARC), to review the analysis in higher detail than the one allowed by presentations at a meeting, and they are tasked with asking you to correct any little fault or unclear point, both in the analysis procedure and in its documentation. These people are volunteers that are chosen by the Publication Committee. I often am part of such ARCs, mostly in the Top, Higgs, and beyond-two-generations (B2G) PAGs.

After the review (that can last a couple of weeks or a few months, depending on many factors), if the ARC is happy, then the ARC chair certifies that the analysis is ready to be published (this is referred to as to be green-lighted), and an approval meeting is scheduled at the PAG.

In this approval meeting, you have to describe again to the PAG the full analysis, together with all the checks requested by the ARC, and in the end a decision is made by the conveners (and sometimes by the physics coordination of the experiment) on whether the analysis is ready. If this is the case, then the analysis is approved.

From this, there are two possible paths.

If the analysis is supposed to be presented at a conference very close in time (e.g. a week from the preapproval), then a public note, called Physics Analysis Summary (PAS) is released immediately to the public. If not, then a draft paper is presented to the whole collaboration of 2000 people, and everyone can, for two weeks, send comments on the paper draft to the authors. This process is called Collaboration Wide Review (CWR).

The papers must adhere to a strict lexical standard, and we have precise guidelines for the style of English we have to use. There is even the role of language editor, which is the person tasked with ensuring the quality of the scientific writing, and to be one you have to get certified by the Publication Committee (I will get my certification with this paper, which is why it needs to be super-polished and adhering to all guidelines, which makes the job longer).

After CWR, you as an author have to answer to each and every comment you have received, and the ARC chair is tasked with certifying when you are done doing so.

After that, the paper is finally released, sent to the Arxiv, and submitted to a journal for peer review!

There are a few notable things about this whole process: one is that it is an excruciatingly long process. This is sometimes due to the complexity of some analyses, that need very thorough additional cross-checks, and sometimes due to random factors. It is also not uncommon that between the public PAS and the published paper pass many months (both because of the amount of textual comments received during the CWR, and because of the long process of journal peer review).

This is the reason why the Arxiv is the best source for the latest results of the large LHC experimental collaborations, whereas journals contain mostly old results. Many people (not only in particle physics) are advocating for a change in the journal scheme.

Another aspect worth mentioning is the amount of review such an analysis receives, compared with other fields.

In other fields, you do your work, together with 2-5 colleagues, you redact the paper with them, and then you send the paper to a journal. Two or three people, the referees, are tasked by the journal to review the paper, and when they are happy the paper is published. This means that an analysis will be reviewed by three people.

An analysis by one of the large HEP collaborations, instead, before even being sent to a journal is already reviewed by 20-50 people, approved by 2-4 people, then reviewed in detail by 4 people, then reviewed again by the 20-50 people and approved by the 2-4 people, then reviewed again by potentially 2000 people (usually some other 10-20 people), and finally sent to the journal.

On one side, this is excruciatingly long and sometimes affected by darker sides of the human nature, but on the other side this is very good: that many details that are crucial for the analysis are so complex that a person external to the experiment cannot understand them without a deep knowledge of the experiment, so the usual journal-managed peer review could not possibly be a guarantee of scientific quality.

You may argue that since all the collaboration signs the paper, it is not a real peer review, but what happens in practice is that, when there is not a huge hurry, the review from the collaboration is very, very hardcore, much more rigorous than the one that a journal can provide.

Still, it’s an excruciatingly long process, and you may easily get bored if you are redacting a paper while sitting in a café on a Saturday afternoon.