Saturday, September 24, 2011

Neutrinos and the future of physics

The scientific community is buzzing, or one would imagine we are, after news of this paper came out last week. Could neutrinos, those mysterious particles which hardly interact with normal matter, really be traveling faster than the speed of light? It was all over the news. The collaboration held a news conference. Hints of another cold fusion fiasco creep into conversations.

For the benefit of those who don't feel like trudging through the 24 page paper, here is a brief summary (with pictures!):

The detector is called OPERA, which stands for "Oscillation Project with Emulsion-tRacking Apparatus" (I know, that should be OPETA, but hey, you decide on the acronym first and then fit the name to it). It was designed to measure neutrinos coming from the super proton synhrotron at CERN. The neutrinos produced (as a byproduct, incidentally) at CERN are mainly muon neutrinos (remember that there are three "flavors" of neutrinos), and OPERA, located just under 500 miles away in the Italian national lab of Gran Sasso, hopes to see tau neutrinos. Neutrinos can spontaneously change flavor (the theory, actually, is not that they discretely change, but that they exist as superpositions of all three flavors in various mixing ratios which "condense" into one type when interacting with matter), and measuring the number of muon neutrinos which change into tau neutrinos will help to determine all sorts of interesting physical things (like whether or not neutrinos have any mass... photons don't, but electrons do). Here's what OPERA looks like:

As an added bonus, the detector measures the time spectra of the neutrinos as they pass through it, and the neutrinos, just like the protons that produced them originally (back at CERN), come in bunches. One of these bunches looks roughly like this:

The red curve is the proton time spectrum (from CERN). The black data points make up the neutrino time spectrum from Gran Sasso.
So after these two time spectra are collected for a given "extraction" (ie, a given bunch), they can be compared. Here's the kicker. The curves can be adjusted to fit one another only if the scientists assume the neutrinos which were observed in the OPERA detector are traveling faster than the speed of light. Not much faster, mind you - only 0.0025% - but over the hundreds of kilometers from start to finish, that amounts to 60 nanoseconds (ns). This number they give with a "six sigma" stamp of approval... which, in a nutshell, means that the 60ns result has only a 0.0000002% chance of being a statistical fluke.

Ok... so we have a result that seems to indicate the neutrinos are traveling faster than the speed of light. What I'd like to do now is not argue whether or not this result is correct, but instead point out a common fallacy, if you will, with regard to physics results which may or may not alter the paper's conclusions.

Fallacy #1: More statistics = better result
The authors of the paper make a big deal of the fact that around 16,000 events went into their data analysis. This is a large number of events, and naively it is true that statistical uncertainty goes down as sample size goes up - for something that follows a gaussian distribution (aka "bell curve"), for example (like the exam grades of a typical physics class), the statistical uncertainty increases with the square root of N, where N is the number of events. This means that for a sample size of 100, the uncertainty is 10 events, or 10%. Increase your sample to 1000 and the uncertainty is 32, or 3%. If you had 16,000 events, your statistical uncertainty is 126 events, or a meager 0.8%! So it appears that more stats does mean better results... but this neglects a very important point, and that point is systematic uncertainty.
There are two types of uncertainty in any experiment, as any student who's taken a basic science lab will tell you. Statistical uncertainty is the uncertainty inherent in the number of samples, but systematic uncertainty is a totally different beast. Systematic uncertainties are those which you introduce - intentionally or not - to the experiment, just by the way you do it. Is there an uncertainty in the length of the ruler you used to measure your experimental distances? Is there an uncertainty in the way you recorded the time? Sometimes, it can be very difficult to account for all of these experimental biases, because you may not even know they exist. Maybe you have some bacteria in a petri dish, and you've drawn a grid on the glass of the petri dish to allow you to measure how much the bacteria are moving. So you check the locations of your bacteria at noon, and again at three, and again on Tuesday. But did you stand in exactly the same location when you measured the bacteria against the grid? Did you lean over the petri dish in precisely the same way? Light bends through glass (refraction, the same reason a straw looks bent in a glass of water), so the angle of your eyes to the surface of the petri dish will change, very slightly, the way you see the grid and the bacteria together. Catch that? Sneaky. Yet another systematic uncertainty.
Systematic uncertainties do not get better with more data points. Systematic uncertainties are completely independent of the number of events in a given experiment. In fact, the whole data set - be it 3 events or 3 million - can suffer the same systematic uncertainty, which can sometimes cancel out, but sometimes shift the entire thing one direction or another. Consider your petri dish bacteria. If you drew your grid on the inside of the dish, before putting the whole thing together and filling it with bacteria, then what you see from the outside is always slightly off from what really exists on the inside. Even if you always stand in the same place and look in the same way, the entire grid may still be slightly off from reality, and this would offset your entire data collection. This potential offset is why, in my field, we always try to do experiments in as many different ways and at as many different labs as possible. It serves as a check. If we do things here at our lab and someone else does the same things at their lab, and our results are always offset from one another, a systematic uncertainty is said to exist between the two labs. Two methods can also suffer from a systematic uncertainty between them, such as measuring a nuclear reaction "forward" (oxygen+alpha->proton+fluorine) and "backward" (proton+fluorine->oxygen+alpha).
There are ways to estimate systematic uncertainties, and the more often you do something, the better you get at it (the more a piece of lab equipment is used, for instance, the better characterized it is). But OPERA has only been running a few years (this may seem like a long time, but for neutrino experiments, it's not). The amount of data they've collected is still being analyzed. So the potential for as-yet-unknown systematic uncertainties certainly exists (the authors of the paper even admit this fact explicitly, saying "the potentially great impact of the result motivates the continuation of our studies in order to investigate possible still unknown systematic effects that could explain the observed anomaly").

Now, to be fair, it's entirely possible that this result is real. While the observation of supernova SN1987a seemed to preclude the possibility of neutrinos traveling faster than light, an earlier result from the MINOS experiment indicated that neutrinos they measured might have been going a bit too fast (that experiment, however, had big enough uncertainties that the neutrinos could have been going light speed). String theory allows for faster-than-light travel through fluctuations in the "quantum foam" of spacetime. General relativity, however, does not.

So here are some more specific notes for the scientifically-minded reader, with regard to the things I feel are likely suspects in the search for systematic uncertainties.
1) The limits set by SN1987a are for a different energy regime and, more importantly, a different neutrino flavor (anti-electron neutrinos were detected during this event, vs muon neutrinos for OPERA and MINOS). There could be a systematic effect between neutrino flavors, as well as neutrino energy (the OPERA result cannot rule this out).
2) The GPS signals used to determine location and timing had to be taken at the surface, while the laboratories are actually far underground. This leads to an extrapolation, which can lead to uncertainties. Was the curvature of the Earth accounted for? The density and type of the rock?
3) Something I feel is most telling - the neutrino time-of-flight (TOF), which is ultimately compared to the time expected if the neutrinos were going the speed of light to get this "delta TOF" of 60ns, is not actually measured. As I mentioned earlier, the proton time spectra and neutrino time spectra are measured within their respective detectors/labs, and timestamped to within a few nanoseconds. In theory, there is no discrepancy between the timestamps (GPS and cesium-clock generated) at the different labs, but it is even emphasized by the authors themselves that this is not a t(stop)-t(start) kind of experiment. There is nothing that's actually starting a clock when the protons are produced and stopping the clock when a neutrino is seen, and that's because the whole process is statistical (they can't know exactly when a given proton will create a given neutrino, or where). So they do a "maximum likelihood" fit (a fancy, mathematical way of saying "we moved the two curves until they overlapped") to the two time spectra to determine how far off they are from each other. What if there's a systematic uncertainty here? It alters the entire result. What if the neutrino bunch just measured corresponds not to the proton bunch you think it does, but to the one before? It's not that the neutrinos are traveling slightly faster than the speed of light, they're traveling slower, and you're just off by one 'cycle.' I didn't get in the preprint a good description of how they know which neutrino bunch corresponds to which proton bunch, other than simply expecting them to be traveling light speed and assuming that anything falling within a small window around that would be real.

One last humorous note, which I mentioned previously on facebook. Have you ever been working in a spreadsheet program, entering a function into cell B2 that depends on cell D7? Everything is fine unless the content of cell D7 also depends on the value in cell B2... then you get what's known as a recursion error. The functions can't be solved because they each depend on the other, so you end up stuck inside an infinite loop (B2's value leads to D7's which leads to B2's which leads to D7's which leads to...). The OPERA result depends (rather heavily) on GPS timing and position signals. But GPS depends on relativity, and relativity, in turn, depends on the speed of light being constant for all observers (that means neutrinos, too). But if the OPERA result is correct, then the neutrinos have traveled faster than the speed of light, contradicting relativity. If your result contradicts the possibility of your result, how can it be your result?

I've heard a lot of good scientists weigh in on this result and its potential consequences. One real (rather philosophical) question remains. Does this mean the end of physics is looming? Hardly. This is science - doing experiments, drawing conclusions, testing those conclusions with more experiments. Overturning long-held (and often dearly loved) hypotheses is part of the deal, so long as it's done right. Time will tell if this is one of those instances... and won't it be great to know you were there when it happened?

The OPERA Collaboraton: T. Adam, N. Agafonova, A. Aleksandrov, O. Altinok, P. Alvarez Sanchez, S. Aoki, A. Ariga, T. Ariga, D. Autiero, A. Badertscher, A. Ben Dhahbi, A. Bertolin, C. Bozza, T. Brugiére, F. Brunet, G. Brunetti, S. Buontempo, F. Cavanna, A. Cazes, L. Chaussard, M. Chernyavskiy, V. Chiarella, A. Chukanov, G. Colosimo, M. Crespi, N. D'Ambrosios, Y. Déclais, P. del Amo Sanchez, G. De Lellis, M. De Serio, F. Di Capua, F. Cavanna, A. Di Crescenzo, D. Di Ferdinando, N. Di Marco, S. Dmitrievsky, M. Dracos, D. Duchesneau, S. Dusini, J. Ebert, I. Eftimiopolous, O. Egorov, A. Ereditato, L. S. Esposito, J. Favier, T. Ferber, R. A. Fini, T. Fukuda, A. Garfagnini, G. Giacomelli, C. Girerd, M. Giorgini, M. Giovannozzi, J. Goldberga, C. Göllnitz, L. Goncharova, Y. Gornushkin, G. Grella, F. Griantia, E. Gschewentner, C. Guerin, A. M. Guler, C. Gustavino, K. Hamada, T. Hara, M. Hierholzer, A. Hollnagel, M. Ieva, H. Ishida, K. Ishiguro, K. Jakovcic, C. Jollet, M. Jones, F. Juget, M. Kamiscioglu, J. Kawada, S. H. Kim, M. Kimura, N. Kitagawa, B. Klicek, J. Knuesel, K. Kodama, M. Komatsu, U. Kose, I. Kreslo, C. Lazzaro, J. Lenkeit, A. Ljubicic, A. Longhin, A. Malgin, G. Mandrioli, J. Marteau, T. Matsuo, N. Mauri, A. Mazzoni, E. Medinaceli, F. Meisel, A. Meregaglia, P. Migliozzi, S. Mikado, D. Missiaen, K. Morishima, U. Moser, M. T. Muciaccia, N. Naganawa, T. Naka, M. Nakamura, T. Nakano, Y. Nakatsuka, D. Naumov, V. Nikitina, S. Ogawa, N. Okateva, A. Olchevsky, O. Palamara, A. Paoloni, B. D. Park, I. G. Park, A. Pastore, L. Patrizii, E. Pennacchio, H. Pessard, C. Pistillo, N. Polukhina, M. Pozzato, K. Pretzl, F. Pupilli, R. Rescigno, T. Roganova, H. Rokujo, G. Rosa, I. Rostovtseva, A. Rubbia, A. Russo, O. Sato, Y. Sato, A. Schembri, J. Schuler, L. Scotto Lavina, J. Serrano, A. Sheshukov, H. Shibuya, G. Shoziyoev, S. Simone, M. Sioli, C. Sirignano, G. Sirri, J. S. Song, M. Spinetti, N. Starkov, M. Stellacci, M. Stipcevic, T. Strauss, P. Strolin, S. Takahashi, M. Tenti, F. Terranova, I. Tezuka, V. Tioukov, P. Tolun, T. Tran, S. Tufanli, P. Vilain, M. Vladimirov, L. Votano, J. -L. Vuilleumier, G. Wilquet, B. Wonsak, J. Wurtz, C. S. Yoon, J. Yoshida, Y. Zaitsev, S. Zemskova, & A. Zghiche (2011). Measurement of the neutrino velocity with the OPERA detector in the CNGS beam arXiv arXiv: 1109.4897v1


  1. Another good post from (it appears Chad beat me to it) here.

  2. That is a really good analysis of the paper (which I have not read) and description of how all the scientists I have talked to have reacted. I just want to link to xkcd to add a humorous example of a physicist dealing with ley people's reactions.

  3. nice blog !! i was looking for blogs related of Physics lab equipment . then i found this blog, this is really nice and interested to read. thanks to author for sharing this type of information.


Think carefully before you post. I reserve the right to moderate any comments posted to my blog.