Misplaced confidence: limits to statistical inference in cyclostratigraphy

David G. Smith

doi:10.21701/bolgeomin.131.2.005

Authors

David G. Smith

DOI:

https://doi.org/10.21701/bolgeomin.131.2.005

Keywords:

cyclostratigraphy, spectral analysis, confidence limits, null hypothesis

Abstract

Spectral (frequency-domain) analysis is used for quantitative confirmation of cyclicity in climate-proxy data. Cyclostratigraphic power spectra are typically accompanied by ‘confidence limits’, whether or not a statistical test has been explicitly invoked. Peaks in spectral power suggest candidate cyclic frequencies; confidence limits (CLs) appear to provide a visual guide to their relative importance, and are conventionally used in a correspondingly informal way. Confidence limits are, however, inseparable from formal tests of statistical significance; they derive from a statistical null hypothesis, and provide a threshold for its acceptance or rejection. In the procedure conventionally used in cyclostratigraphy (and implemented in several specialised software packages), noise models and confidence limits are generated automatically. Although the user may be unaware of it, the null hypothesis on which these CLs are based is calibrated for a (confirmatory) test of significance at exactly one frequency. Extending their application to an exploratory search of spectral peaks at all frequencies is statistically inadmissible. Debate over the role and correct calculation of CLs in cyclostratigraphy remains unresolved: this contribution seeks to clarify the disagreement over their use by explaining the role of CLs in statistical significance tests generally, and comparing it with their conventional use in cyclostratigraphy. Through examples of the correct and incorrect use of the conventional method, I show that the customary informal use of statistical test criteria cannot be sustained. Significance thresholds cannot be calculated in most cases; wrongly estimated confidence limits lead to false positive cycle identifications, with adverse consequences for calibration of the geological time scale.

Downloads

Download data is not yet available.

References

Andrews, S.D., Cornwell, D.G., Trewin, N.H., Hartley, A.J. and Archer, S.G., 2018. Reply to the discussion on 'A 2.3 Million Year Lacustrine Record of Orbital Forcing from the Devonian of Northern Scotland' (Journal of the Geological Society, 173, 474-488). Journal of the Geological Society, 175, 563, 1 December 2017. London. https://doi.org/10.1144/jgs2017-132

Bailey, R. J. 2009. Cyclostratigraphic reasoning and orbital time calibration. Terra Nova, 21(5), 340-351. https://doi.org/10.1111/j.1365-3121.2009.00890.x

Benjamini, Y. and Hochberg, Y., 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289-300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x

Carp, J., 2012. The secret lives of experiments: methods reporting in the fMRI literature. Neuroimage, 63(1), 289-300. https://doi.org/10.1016/j.neuroimage.2012.07.004

Crampton, J.S., Meyers, S.R., Cooper, R.A., Sadler, P.M., Foote, M. and Harte, D., 2018. Pacing of Paleozoic macroevolutionary rates by Milankovitch grand cycles. Proceedings of the National Academy of Sciences, 115(22), 5686-5691. https://doi.org/10.1073/pnas.1714342115

Da Silva, A.C., Dekkers, M.J., De Vleeschouwer, D., Hladil, J., Chadimova, L., Slavík, L. and Hilgen, F.J. 2019. Millennial-scale climate changes manifest Milankovitch combination tones and Hallstatt solar cycles in the Devonian greenhouse world (Forum Reply). Geology, 47(10), 489-490. https://doi.org/10.1130/G46732Y.1

Gelman, A. and Loken, E., 2014. The statistical crisis in science: data-dependent analysis - a "garden of forking paths" - explains why many statistically significant comparisons don't hold up. American Scientist, 102(6), 460-466. https://doi.org/10.1511/2014.111.460

Gong, Z. and Kodama, K.P., 2018. Reply to the comment on "Rock magnetic cyclostratigraphy of the Doushantuo Formation, South China and its implications for the duration of the Shuram carbon isotope excursion". (Gong, Z., Kodama, K.P. and Li, Y.X. Precambrian Research, 289 (2017), 62-74). Precambrian Research, 310, 467-470. https://doi.org/10.1016/j.precamres.2018.03.008

Hilgen, F.J., Hinnov, L.A., Aziz, H.A., Abels, H.A., Batenburg, S., Bosmans, J.H., de Boer, B., Hüsing, S.K., Kuiper, K.F., Lourens, L.J. and Rivera, T., 2015. Stratigraphic continuity and fragmentary sedimentation: the success of cyclostratigraphy as part of integrated stratigraphy. Geological Society Special Publications, 404(1), 157-197. London. https://doi.org/10.1144/SP404.12

Hinnov, L.A., Ruhl, M.R. and Hesselbo, S.P., 2018. Reply to the Comment on "Astronomical constraints on the duration of the Early Jurassic Pliensbachian Stage and global climatic fluctuations" (Ruhl et al., Earth and Planetary Science Letters, 455 (2016) 149-165). Earth and Planetary Science Letters, 481, 415-419. https://doi.org/10.1016/j.epsl.2017.10.061

Hinnov, L.A., Wu, H. and Fang, Q., 2016. Reply to the comment on "Geologic evidence for chaotic behavior of the planets and its constraints on the third-order eustatic sequences at the end of the Late Paleozoic Ice Age" by Qiang Fang, Huaichun Wu, Linda A. Hinnov, Xiuchun Jing, Xunlian Wang, and Qingchun Jiang (Palaeogeography Palaeoclimatology Palaeoecology 400 (2015) 848-859). Palaeogeography, Palaeoclimatology, Palaeoecology, 461, 475-480. https://doi.org/10.1016/j.palaeo.2016.07.030

Howe, T.S., Corcoran, P.L., Longstaffe, F.J., Webb, E.A. and Pratt, R.G., 2018. Response to the discussion on "Climatic cycles recorded in glacially influenced rhythmites of the Gowganda Formation, Huronian Supergroup" (Precambrian Research, 286 (2016), 269-280). Precambrian Research, 316, 327. https://doi.org/10.1016/j.precamres.2018.07.019

Ioannidis, J.P., 2005. Why most published research findings are false. PLos Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124

Mann, M.E. and Lees, J.M., 1996. Robust estimation of background noise and signal detection in climatic time series. Climatic Change, 33(3), 409-445. https://doi.org/10.1007/BF00142586

Meyers, S.R., 2019a. Astrochron: An R package for astrochronology (version 0.9). https://cran.r-project.org/web/packages/astrochron/index.html

Meyers, S.R., 2019b. Cyclostratigraphy and the problem of astrochronologic testing. Earth-Science Reviews, 190, 190-223. https://doi.org/10.1016/j.earscirev.2018.11.015

Meyers, S.R. and Sageman, B.B., 2007. Quantification of deep-time orbital forcing by average spectral misfit. American Journal of Science, 307(5), 773-792. https://doi.org/10.2475/05.2007.01

Munafò, M.R., Nosek, B.A., Bishop, D.V., Button, K.S., Chambers, C.D., Du Sert, N.P., Simonsohn, U., Wagenmakers, E.J., Ware, J.J. and Ioannidis, J.P., 2017. A manifesto for reproducible science. Nature Human Behaviour, 1(1), 1-9. https://doi.org/10.1038/s41562-016-0021

National Academies of Sciences, Engineering, and Medicine, 2019. Reproducibility and Replicability in Science. National Academies Press, Washington DC. Nuzzo, R., 2014. Scientific method: statistical errors. Nature News, 506 (7487), 150. https://doi.org/10.1038/506150a

Nuzzo, R., 2015. How scientists fool themselves - and how they can stop. Nature, 526 (7572), 182-185. https://doi.org/10.1038/526182a

Ruhl, M., Hesselbo, S.P., Hinnov, L., Jenkyns, H.C., Xu, W., Riding, J.B., Storm, M., Minisini, D., Ullmann, C.V. and Leng, M.J., 2016. Astronomical constraints on the duration of the Early Jurassic Pliensbachian Stage and global climatic fluctuations. Earth and Planetary Science Letters, 455, 149-165. https://doi.org/10.1016/j.epsl.2016.08.038

Schwarzacher, W., 1975. Sedimentation Models and Quantitative Stratigraphy. Developments in Sedimentology, 19. Elsevier, 382 pp.

Simmons, J.P., Nelson, L.D. and Simonsohn, U., 2011. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359-1366. https://doi.org/10.1177/0956797611417632

Smith, D.G., 2019. Millennial-scale climate changes manifest Milankovitch combination tones and Hallstatt solar cycles in the Devonian greenhouse world (Forum Comment: Da Silva, A.C., Dekkers, M.J., De Vleeschouwer, D., et al., Geology, 47 (2019), 489-490). Geology, 47(10), 488. https://doi.org/10.1130/G46732Y.1

Smith, D.G. and Bailey, R.J., 2017a. Discussion on 'A 2.3 million year lacustrine record of orbital forcing from the Devonian of northern Scotland' (Journal of the Geological Society, 173, 474-488). Journal of the Geological Society, 175, 561-562. London. https://doi.org/10.1144/jgs2016-137

Smith, D.G. and Bailey, R.J., 2017b. Discussion on 'Orbital calibration of the late Campanian carbon isotope event in the North Sea' (Journal of the Geological Society, 173, 504-517). Journal of the Geological Society, 175, 564-565. London. https://doi.org/10.1144/jgs2016-148

Smith, D.G. and Bailey, R.J., 2018a. Comment on "Astronomical constraints on the duration of the Early Jurassic Pliensbachian Stage and global climatic fluctuations" (Earth and Planetary Science Letters 455 (2016), 149-165). Earth and Planetary Science Letters, 481, 412-414. https://doi.org/10.1016/j.epsl.2017.10.059

Smith, D.G. and Bailey, R.J., 2018b. Discussion: Howe, T.S., Corcoran, P.L., Longstaffe, F.J., Webb, E.A., & Pratt, R.G. (2016). Climatic cycles recorded in glacially influenced rhythmites of the Gowganda Formation, Huronian Supergroup (Precambrian Research, 286, (2016), 269-280). Precambrian Research, 316, 324-326. https://doi.org/10.1016/j.precamres.2017.04.022

Smith, D.G. and Bailey, R.J., 2018c. Comment on "Rock magnetic cyclostratigraphy of the Doushantuo Formation, South China and its implications for the duration of the Shuram carbon isotope excursion". (Gong, Z., Kodama, K.P. and Li, Y.X., Precambrian Research, 289 (2017), 62-74). Precambrian Research 310, 463-466. https://doi.org/10.1016/j.precamres.2017.04.016

Smith, D.G., Bailey, R.J. and Vaughan, S., 2016. Comment on "Geologic evidence for chaotic behavior of the planets and its constraints on the third-order eustatic sequences at the end of the Late Paleozoic Ice Age" by Fang, Q., Wu, H., Hinnov, LA, Jing, X., Wang, X., and Jiang, Q. (Palaeogeography, Palaeoclimatology, Palaeoecology, 440 (2015), 848-859). Palaeogeography, Palaeoclimatology, Palaeoecology, 461, 472-474. https://doi.org/10.1016/j.palaeo.2016.04.040

Thibault, N. and Perdiou, A. 2018. Reply to the discussion on 'Orbital calibration of the late Campanian carbon isotope event in the North Sea' (Journal of the Geological Society, 173 (2016), 504-517). Journal of the Geological Society, 175, 566-567, 2018. London. https://doi.org/10.1144/jgs2018-001

Tukey, John W. 1977, Exploratory Data Analysis, Reading, Mass. Addison-Wesley. Vaughan, S., 2013. Scientific Inference: Learning from Data. Cambridge University Press, 224 pp.

Vaughan, S., 2010. A Bayesian test for periodic signals in red noise. Monthly Notices of the Royal Astronomical Society, 402(1), 307-320. https://doi.org/10.1111/j.1365-2966.2009.15868.x

Vaughan, S., 2013. Scientific Inference: Learning from Data. Cambridge University Press, 224 pp. https://doi.org/10.1017/CBO9781139176071

Vaughan, S., Bailey, R.J. and Smith, D.G. 2011. Detecting cycles in stratigraphic data: spectral analysis in the presence of red noise. Paleoceanography 26. https://doi.org/10.1029/2011PA002195

Weedon, G.P., 2003. Time-Series Analysis and Cyclostratigraphy: Examining Stratigraphic Records of Environmental Cycles. Cambridge University Press, 259 pp. https://doi.org/10.1017/CBO9780511535482

Weedon, G.P., Page, K.N. and Jenkyns, H.C., 2019. Cyclostratigraphy, stratigraphic gaps and the duration of the Hettangian Stage (Jurassic): insights from the Blue Lias Formation of southern Britain. Geological Magazine, 156 (9), 1469-1509. https://doi.org/10.1017/S0016756818000808

Weedon, G.P., 2020. Confirmed detection of Palaeogene and Jurassic orbitally-forced sedimentary cycles in the depth domain using False Discovery Rates and Bayesian probability spectra. Boletin Geológico y Minero, 131, 2, 207-230. https://doi.org/10.21701/bolgeomin.131.2.001