sunsettommy
11-15-2005, 08:17 PM
From PROMETHEUS,
November 14, 2005
Why Does the Hockey Stick Debate Matter?
Post by Ross McKitrick
Roger Pielke Jr. has posed a challenge to Michael Mann and us to briefly explain why each of us thinks the ongoing hockey stick debate matters.
Another excerpt:
So: why does it matter?
1. It matters because it concerns the validity of an influential scientific paper. Mann’s 1998 and 1999 papers (which I’ll call “MBH”) have been heavily cited and highly influential. The paleoclimate field seems to have organized itself around them: other papers since then have gained prominence in proportion as they appear to back up MBH, whereas papers that contradict it have little prospect of being published or are relegated to lower-profile outlets. A popular icon in paleoclimate circles these days is what can be called a “spaghetti graph,” showing a pastiche of climate reconstructions from a small group of authors who call themselves the “Hockey Team”. They agree on few details, other than that the Medieval Warm Period is not as warm as the 20th Century.
Yet MBH turns out to have major flaws that fundamentally undermine its conclusions. These issues are interesting in their own right because MBH is a famous paper. But they also have wider scientific implications. MBH was “helped” along to its conclusions by some very convenient decisions about small changes to methodology, small edits to data series and not-so-small decisions about using contaminated bristlecone data. Maybe some of the other studies that appear to confirm MBH were also “helped” along so they would appear to agree with it. Efforts to evaluate the whole spaghetti graph has encountered maddening secrecy by the other authors concerning their data and methods, just as with MBH. But enough has been discovered to support a couple of assertions.
(a) The other spaghetti graph diagrams lack robustness. They all depend on delicate editing of weak data and just-so methodology. None are al dente: these are very soft noodles, and a plateful of weak results does not add up to a strong conclusion.
(b) There is an unexamined problem of spurious statistics in multiproxy constructions. Hockey team methods mine autocorrelated proxy data for simple correlations with autocorrelated temperature data. It is a classic recipe for spurious results, as has long been known in econometrics following the seminar studies of Granger, Engle and Phillips. What was predictable on theoretical grounds is now emerging empirically: proxies that extend past 1980 have no explanatory power for recent temperatures. And by implication, the existing corpus of multiproxy studies provides spurious information about the historical climate. Despite occasional claims of technical rigour, none of the spaghetti graph lines come from papers where the spurious regression problem was dealt with.
2. It matters because it exposes the uncomfortable reality about journal peer review. MBH(98) was published in Nature, considered by some the world’s “leading” scientific journal. Nature never verified that data were correctly listed: as it happens they weren’t. Nature never verified that data archiving rules were followed: they weren’t. Nature never verified that methods were accurately stated: they weren’t. Nature never verified that stated methods yield the stated results: they don’t. Nature undertook only minimal corrections to its publication record after notification of these things, and even allowed authors to falsely claim that their omissions on these things didn’t affect their published results.
In light of this, it is far past time for a wide-ranging discussion on what ‘peer review’ actually is. Policymakers routinely appeal to it as some kind of quality assurance guarantee. But obviously it isn’t. It serves some purpose internal to the world of scientific publishing, but policymakers’ beliefs about what peer review guarantees are for the most part sheer fantasy.
3. It matters because it exposes the uncomfortable reality about the IPCC. The IPCC’s use of the hockey stick was not incidental: it is prominent throughout the 2001 report. Yet they did not subject it to any independent checking: revealing an astonishingly cavalier attitude to the quality of their case. This raises the question of whether anything in the report was subject to serious, independent checking. They allowed chapter authors to heavily promote their own work with little or no oversight. They published false claims about the hockey stick’s statistical robustness and have never made any effort to retract them. On the basis of the MBH claims, their 2001 report reversed their 1990 conclusions about the MWP, and over-rode their 1995 warnings about not relying on bristlecone data, in order to promote the conclusions implied by the hockey stick. They encouraged governments around the world to rely heavily on a graph they themselves had not independently checked. One reason the hockey stick debate matters is because it exposes as worthless the guarantees given up to now about why the world should rely on the IPCC.
In this light I have no patience for the reaction by scientists to the Barton investigation. Why shouldn’t legislators begin asking questions about how the IPCC (and its allies in the science community) produce their reports? Policymakers have strong evidence that the IPCC process did not actually involve the rigorous checks and balances that they boasted of when releasing their 2001 report. It would be negligent of lawmakers not to open a wide-ranging investigation of this. Anyone who thinks the Barton investigation is unnecessary must think that IPCC reports don’t really matter: but they do, which is one reason why the hockey stick debate also does.
4. It matters because it exposes the uncomfortable reality about how governments use scientific information Canada (and many other countries) used the hockey stick heavily in their promotion of the Kyoto Accord. It is still prominent in government publications. Canada boasts of having spent $3 billion on climate change initiatives, much of it going to research. Yet for all the billions of dollars spent, and for all the proliferation of staff working on the matter, no one in government checked the hockey stick. Even when Canada’s chief climate science advisor and the Prime Minister’s own scientific advisor were personally informed about flaws in the hockey stick, no effort was made to remedy the government’s error. We have never been contacted by a single federal government scientist or other staff member for information on this topic, even though Environment Canada has in the past made heavy use of the hockey stick and more recently has issued communications supposedly providing “expert” commentary on my work, commentary that is predictably fallacious. Governments apparently use science when it suits them, as a promotional policy tool, with little regard to the facts of the matter. Perhaps it is naïve of me to have expected otherwise, but the realization still disappoints.
5. It matters because it exposes an uncomfortable reality about the culture of climate science. It took two outsiders to do all this work. Climate scientists in the field ignored the glaring problems in MBH for five years, and only seemed to get engaged after Stephen McIntyre and I began publishing our work. Since then the “engagement” of climate scientists has primarily consisted of ridicule, nitpicking, obstruction and catcalls from prominent scientists, especially those involved with the IPCC and the University Corporation for Atmospheric Research on Colorado. The few who have offered support tend to do so privately or anonymously.
More here,
http://sciencepolicy.colorado.edu/prometheus/archives/climate_change/000631why_does_the_hockey_.html
__________________________________________________ ____________
Read the comments posted in reaction to the above commentary.It is a hoot!
November 14, 2005
Why Does the Hockey Stick Debate Matter?
Post by Ross McKitrick
Roger Pielke Jr. has posed a challenge to Michael Mann and us to briefly explain why each of us thinks the ongoing hockey stick debate matters.
Another excerpt:
So: why does it matter?
1. It matters because it concerns the validity of an influential scientific paper. Mann’s 1998 and 1999 papers (which I’ll call “MBH”) have been heavily cited and highly influential. The paleoclimate field seems to have organized itself around them: other papers since then have gained prominence in proportion as they appear to back up MBH, whereas papers that contradict it have little prospect of being published or are relegated to lower-profile outlets. A popular icon in paleoclimate circles these days is what can be called a “spaghetti graph,” showing a pastiche of climate reconstructions from a small group of authors who call themselves the “Hockey Team”. They agree on few details, other than that the Medieval Warm Period is not as warm as the 20th Century.
Yet MBH turns out to have major flaws that fundamentally undermine its conclusions. These issues are interesting in their own right because MBH is a famous paper. But they also have wider scientific implications. MBH was “helped” along to its conclusions by some very convenient decisions about small changes to methodology, small edits to data series and not-so-small decisions about using contaminated bristlecone data. Maybe some of the other studies that appear to confirm MBH were also “helped” along so they would appear to agree with it. Efforts to evaluate the whole spaghetti graph has encountered maddening secrecy by the other authors concerning their data and methods, just as with MBH. But enough has been discovered to support a couple of assertions.
(a) The other spaghetti graph diagrams lack robustness. They all depend on delicate editing of weak data and just-so methodology. None are al dente: these are very soft noodles, and a plateful of weak results does not add up to a strong conclusion.
(b) There is an unexamined problem of spurious statistics in multiproxy constructions. Hockey team methods mine autocorrelated proxy data for simple correlations with autocorrelated temperature data. It is a classic recipe for spurious results, as has long been known in econometrics following the seminar studies of Granger, Engle and Phillips. What was predictable on theoretical grounds is now emerging empirically: proxies that extend past 1980 have no explanatory power for recent temperatures. And by implication, the existing corpus of multiproxy studies provides spurious information about the historical climate. Despite occasional claims of technical rigour, none of the spaghetti graph lines come from papers where the spurious regression problem was dealt with.
2. It matters because it exposes the uncomfortable reality about journal peer review. MBH(98) was published in Nature, considered by some the world’s “leading” scientific journal. Nature never verified that data were correctly listed: as it happens they weren’t. Nature never verified that data archiving rules were followed: they weren’t. Nature never verified that methods were accurately stated: they weren’t. Nature never verified that stated methods yield the stated results: they don’t. Nature undertook only minimal corrections to its publication record after notification of these things, and even allowed authors to falsely claim that their omissions on these things didn’t affect their published results.
In light of this, it is far past time for a wide-ranging discussion on what ‘peer review’ actually is. Policymakers routinely appeal to it as some kind of quality assurance guarantee. But obviously it isn’t. It serves some purpose internal to the world of scientific publishing, but policymakers’ beliefs about what peer review guarantees are for the most part sheer fantasy.
3. It matters because it exposes the uncomfortable reality about the IPCC. The IPCC’s use of the hockey stick was not incidental: it is prominent throughout the 2001 report. Yet they did not subject it to any independent checking: revealing an astonishingly cavalier attitude to the quality of their case. This raises the question of whether anything in the report was subject to serious, independent checking. They allowed chapter authors to heavily promote their own work with little or no oversight. They published false claims about the hockey stick’s statistical robustness and have never made any effort to retract them. On the basis of the MBH claims, their 2001 report reversed their 1990 conclusions about the MWP, and over-rode their 1995 warnings about not relying on bristlecone data, in order to promote the conclusions implied by the hockey stick. They encouraged governments around the world to rely heavily on a graph they themselves had not independently checked. One reason the hockey stick debate matters is because it exposes as worthless the guarantees given up to now about why the world should rely on the IPCC.
In this light I have no patience for the reaction by scientists to the Barton investigation. Why shouldn’t legislators begin asking questions about how the IPCC (and its allies in the science community) produce their reports? Policymakers have strong evidence that the IPCC process did not actually involve the rigorous checks and balances that they boasted of when releasing their 2001 report. It would be negligent of lawmakers not to open a wide-ranging investigation of this. Anyone who thinks the Barton investigation is unnecessary must think that IPCC reports don’t really matter: but they do, which is one reason why the hockey stick debate also does.
4. It matters because it exposes the uncomfortable reality about how governments use scientific information Canada (and many other countries) used the hockey stick heavily in their promotion of the Kyoto Accord. It is still prominent in government publications. Canada boasts of having spent $3 billion on climate change initiatives, much of it going to research. Yet for all the billions of dollars spent, and for all the proliferation of staff working on the matter, no one in government checked the hockey stick. Even when Canada’s chief climate science advisor and the Prime Minister’s own scientific advisor were personally informed about flaws in the hockey stick, no effort was made to remedy the government’s error. We have never been contacted by a single federal government scientist or other staff member for information on this topic, even though Environment Canada has in the past made heavy use of the hockey stick and more recently has issued communications supposedly providing “expert” commentary on my work, commentary that is predictably fallacious. Governments apparently use science when it suits them, as a promotional policy tool, with little regard to the facts of the matter. Perhaps it is naïve of me to have expected otherwise, but the realization still disappoints.
5. It matters because it exposes an uncomfortable reality about the culture of climate science. It took two outsiders to do all this work. Climate scientists in the field ignored the glaring problems in MBH for five years, and only seemed to get engaged after Stephen McIntyre and I began publishing our work. Since then the “engagement” of climate scientists has primarily consisted of ridicule, nitpicking, obstruction and catcalls from prominent scientists, especially those involved with the IPCC and the University Corporation for Atmospheric Research on Colorado. The few who have offered support tend to do so privately or anonymously.
More here,
http://sciencepolicy.colorado.edu/prometheus/archives/climate_change/000631why_does_the_hockey_.html
__________________________________________________ ____________
Read the comments posted in reaction to the above commentary.It is a hoot!