Filter.shared creates wildly incorrect OtuRare totals

Mothur v.1.41.0 has introduced an error into filter.shared. The command removes the correct number of OTUs based on the criterion given, but incorrectly calculates the OtuRare abundance. This can cause serious analysis errors downstream because it can drastically change OTU relative abundances for ALL Otus (not just the rare OTU) within samples. The number of reads attributed (incorrectly) to OtuRare, for any given sample, is always either 0 or the number of OTUs removed from the dataset (the number of OTUs deemed to be rare). (In some cases it is a multiple of the number of OTUs. For example, if there are 5 rare OTUs, the number of rare reads in OtuRare will be either 0, 5, or a multiple of 5 for any given sample.) In the following example, I made a test data set of 10 OTUs, of which 4 OTUs are singletons. The singleton OTUs should be removed by filter.shared with criterion mintotal=2. Looking at the .filter.shared output file, we see that filter.shared correctly removed the 4 OTUs. However, it incorrectly added up the number of rare reads in each sample: the filtered shared file shows that just one sample was responsible for ALL of the reads in rare OTUs. I re-ran the command, using the same test set, using the previous release of mothur (v1.40): in that case the command runs correctly.

I am allowed only 1 link here, so linked below is the input shared file, and the output (filtered) shared file showing the error. I paste the text of the logfile at the end of the post.

shared file (input for filter.shared)

filtered shared file (output from filter.shared)

and pasted here is the text of the logfile:
Mac version

Using ReadLine

Using Boost

Running 64Bit Version

mothur v.1.41.0
Last updated: 9/27/18
by
Patrick D. Schloss

Department of Microbiology & Immunology

University of Michigan

When using, please cite:
Schloss, P.D., et al., Introducing mothur: Open-source, platform-independent, community-supported software for describing and comparing microbial communities. Appl Environ Microbiol, 2009. 75(23):7537-41.

Distributed under the GNU General Public License

Type ‘help()’ for information on the commands that are available

For questions and analysis support, please visit our forum at

Type ‘quit()’ to exit program

Interactive Mode

mothur > set.dir(input=/Volumes/JetDrive/201807_SFZoo_mothur/inputs,output=/Volumes/JetDrive/201807_SFZoo_mothur/outputs,tempdefault=/Users/Andrea/Documents/mothur)
Mothur’s directories:
outputDir=/Volumes/JetDrive/201807_SFZoo_mothur/outputs/
inputDir=/Volumes/JetDrive/201807_SFZoo_mothur/inputs/
tempDefault=/Users/Andrea/Documents/mothur/

mothur > filter.shared(shared=test.filtershared.shared, mintotal=2, label=1, makerare=t)
1

Removed 4 OTUs.

Output File Names:
/Volumes/JetDrive/201807_SFZoo_mothur/outputs/test.filtershared.1.filter.shared

mothur > quit

Thank you for taking the time to post such a detailed bug report. I have fixed the issue and the change will be part of our official 1.41.0 release coming later this week. You can see the changes here, Fixes rareOtu bug in filter.shared · mothur/mothur@d4667c6 · GitHub.

Thank you for attending to this so quickly!

Andrea