venn(calc=sharedchao) yields lower unshared OTU counts than calc=sharedsobs. Is this correct?

Hello,

I’m a novice with ecology stats and Mothur’s implementation thereof, so I’m hoping someone can answer two questions for me:

1. How is the ‘sharedchao’ calculation applied to unshared OTUs in Mothur’s Venn() command?
2. Why would my unshared OTU estimates decrease when comparing sharedchao to sharedobs?

I've run Mothur's venn() command on three groups with the following command: ```text mothur v.1.35.0 Last updated: 03/23/2014 ... mothur > venn(shared=file.shared, calc=sharedsobs-sharedchao, nseqs=T) ``` As expected, the sharedchao outputs (mostly) show higher shared OTU estimates between the three groups.

However, I didn’t expect the number of unshared OTUs to decrease in the sharedchao output, and I don’t even understand what calculations Mothur may be using to adjust the unshared OTU estimate. I’ve reviewed the sharedchao wiki page (http://www.mothur.org/wiki/Sharedchao), and the “Chao1 shared” equations in the SPADE User guide, but I’m still confused because the terms for shared chao (shared singletons and doubletons) don’t seem to apply to a batch of unshared OTUs. What am I missing?

For reference, my sharedobs OTUs were as follows: A=42 B=28 C=23 AB shared=28 AC shared=26 BC shared=14 ABC shared=70

My sharedchao OTUs were as follows:
A=26.6
B=27.5
C=38.4
AB shared=41.9
AC shared=25.1
BC shared=10.8
ABC shared=114.4

Thanks

Can you send a copy of file.shared to mothur.bugs@gmail.com and provide a link to this thread? We can take a look…