# Robustness and Reproducibility in the Demarcation of OTUs **URL:** \ **Category:** Journal club\ **Created:** [September 1, 2014, 5:42pm UTC](https://forum.mothur.org/t/robustness-and-reproducibility-in-the-demarcation-of-otus/2000 "2014-09-01T17:42:52Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Sebastian](https://avatars.discourse-cdn.com/v4/letter/s/4da419/32.png) [@Sebastian](https://forum.mothur.org/u/Sebastian)\ **Post date:** [September 1, 2014, 5:42pm UTC](https://forum.mothur.org/t/robustness-and-reproducibility-in-the-demarcation-of-otus/2000/1 "2014-09-01T17:42:52Z")
Dear mothur community, I’d like to (again) shamelessly advertise a paper that has just been published online, and which may be of some interest to some of you: Schmidt TSB, Matias Rodrigues JF, von Mering C (2014). Limits to Robustness and Reproducibility in the Demarcation of Operational Taxonomic Units. Environ Microbiol, doi:10.1111/1462-2920.12610 [http://onlinelibrary.wiley.com/doi/10.1111/1462-2920.12610/abstract](http://onlinelibrary.wiley.com/doi/10.1111/1462-2920.12610/abstract) > > > The demarcation of operational taxonomic units (OTUs) from complex sequence data sets is a key step in contemporary studies of microbial ecology. However, as biologically motivated 'optimal' OTU-binning algorithms remain elusive, many conceptually distinct approaches continue to be used. Using a global data set of 887 870 bacterial 16S rRNA gene sequences, we objectively quantified biases introduced by several widely employed sequence clustering algorithms. We found that OTU-binning methods often provided surprisingly non-equivalent partitions of identical data sets, notably when clustering to the same nominal similarity thresholds; and we quantified the resulting impact on ecological data description for a well-defined human skin microbiome data set. We observed that some methods were very robust to varying clustering thresholds, while others were found to be highly susceptible even to slight threshold variations. Moreover, we comprehensively quantified the impact of the choice of 16S rRNA gene subregion, as well as of data set scope and context on algorithm performance. Our findings may contribute to an enhanced comparability of results across sequence-processing pipelines, and we arrive at recommendations towards higher levels of standardization in established workflows. We have investigated six different methods for ‘de novo’ OTU demarcation: hierarchical average linkage (AL), complete linkage (CL), single linkage (SL) clustering, as well as CD-HIT, UCLUST and UPARSE. Our basic question was: how comparable (or not) are the results provided by these methods? When analyzing a 16S dataset, how much interpretation bias is introduced by the choice of clustering method alone? How robust / sensitive is clustering to minor parameter changes, such as minute threshold changes, or changes in clustering context? How similar are clusterings based on subregions only when compared to full-length clusterings? We ran a series of tests to look into these questions. We focused on cluster composition to measure similarity between partitions – do two given methods tend to bin the same sequences together, or not? Our results indicate that AL and CL, and somewhat surprisingly also CD-HIT provided highly robust and reproducible clustering, whereas SL, UCLUST and UPARSE were more sensitive to even slight parameter changes. For example, simply by moving e.g. from a 97% cutoff to 97.2% led to significantly different clusterings for the latter heuristics. We observed that clustering for all methods was generally ‘replicable’ – when repeating the exact same clustering run, results were replicated. However, not all methods were also ‘reproducible’ – that is, they did not necessarily provide concordant results under slightly changed computational setups. For more details, feel free to take a look at the manuscript 😉 I am posting this here, because we are genuinely interested in getting feedback by you guys – I guess the mothur community is the right ‘target audience’ for this kind of studies. So if you have any questions, please feel free to ask 🙂 Best, Sebastian ---
**Author:** ![patientwind](https://avatars.discourse-cdn.com/v4/letter/p/3da27b/32.png) [@patientwind](https://forum.mothur.org/u/patientwind)\ **Post date:** [February 11, 2015, 12:02am UTC](https://forum.mothur.org/t/robustness-and-reproducibility-in-the-demarcation-of-otus/2000/2 "2015-02-11T00:02:42Z")
Can’t find your paper. Not with the links provided here, not on Wiley, or Web of Science. Not even with a google search. FYI ---
**Author:** ![pschloss](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.mothur.org/pschloss/32/4_2.png) [@pschloss](https://forum.mothur.org/u/pschloss)\ **Post date:** [February 11, 2015, 2:14pm UTC](https://forum.mothur.org/t/robustness-and-reproducibility-in-the-demarcation-of-otus/2000/3 "2015-02-11T14:14:41Z")
Just checked and it worked for me.