# Making OTUs without distance matrix **URL:** \ **Category:** Theory behind mothur\ **Created:** [September 19, 2019, 11:33am UTC](https://forum.mothur.org/t/making-otus-without-distance-matrix/20111 "2019-09-19T11:33:25Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![mafernandez](https://avatars.discourse-cdn.com/v4/letter/m/45deac/32.png) [@mafernandez](https://forum.mothur.org/u/mafernandez)\ **Post date:** [September 19, 2019, 11:33am UTC](https://forum.mothur.org/t/making-otus-without-distance-matrix/20111/1 "2019-09-19T11:33:25Z")
Hi there. I am following MiSeq SOP with some variations in the pipeline to better suit my needs and I face the same problem time after time: my distance matrix is too big to fit into memory/RAM of my computer and cluster() or even cluster.split() fail. Following the suggestions exposed here to my initial doubt: > [@Problems when using cluster.split on huge .dist file](https://forum.mothur.org/t/problems-when-using-cluster-split-on-huge-dist-file/20073): > > Hello there. This is a continuation of the topic posted at: As it has been closed due to inactivity. I have a problem when trying to cluster eukaryotic sequences using either cluster or cluster.split. After several tries where the system collapsed I ran dist.seqs with the column output option and then tried to run cluster.split like this: cluster.split(column=Eukarya\_rocas.trim.contigs.good.unique.good.filter.unique.precluster.pick.dist, count=Eukarya\_rocas.trim.contigs.good.unique.good.f… I tried to make an approximation based on taxonomy instead of distance to run the cluster.split with the ‘file=’ option. However, it is also taking so long… Therefore, I decided to make an approximation to my data without using OTUs, just using taxonomic information coming from ‘classify.seqs’. My only problem would appear if I need to claculate richness index, but I think I could solve that. And here is my question: If you follow MiSeq SOP you reach the step of ‘pre.cluster’, when you ‘merge’ sequences that are 1 nt each 100bp different. If after doing this, you repeat the command but asuming a difference of 3 nt each 100bp, would it be equivalent to make OTUs at a cutoff=0.03 using a distance matrix? Would it be correct to treat these output sequences as OTUs for subsequent analyses? Thanks ---
**Author:** ![pschloss](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.mothur.org/pschloss/32/4_2.png) [@pschloss](https://forum.mothur.org/u/pschloss)\ **Post date:** [September 19, 2019, 1:31pm UTC](https://forum.mothur.org/t/making-otus-without-distance-matrix/20111/2 "2019-09-19T13:31:26Z")
If you do 3/100 then you’re really making 6% OTUs since one sequence could be 3 off of two other sequences and those sequences would then be 6 off of each other. Are you using the latest version of mothur? We have put in some new algorithms that use a smaller distance matrix and use less RAM. Pat ---
**Author:** ![mafernandez](https://avatars.discourse-cdn.com/v4/letter/m/45deac/32.png) [@mafernandez](https://forum.mothur.org/u/mafernandez)\ **Post date:** [September 19, 2019, 1:39pm UTC](https://forum.mothur.org/t/making-otus-without-distance-matrix/20111/3 "2019-09-19T13:39:59Z")
I see… So, if I set pre.cluster to ndiffs=1/100bp is like having 98% similarity OTUs? This would work also… Yes, I am using mothur v1.42.3 on a Windows server with 16 processors and 64gb of RAM, but my .dist files are over 70gb, so I think this is the main problem. Thanks ---
**Author:** ![pschloss](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.mothur.org/pschloss/32/4_2.png) [@pschloss](https://forum.mothur.org/u/pschloss)\ **Post date:** [September 19, 2019, 2:05pm UTC](https://forum.mothur.org/t/making-otus-without-distance-matrix/20111/4 "2019-09-19T14:05:20Z")
Are you using `cutoff=0.03` in `dist.seqs`? ---
**Author:** ![mafernandez](https://avatars.discourse-cdn.com/v4/letter/m/45deac/32.png) [@mafernandez](https://forum.mothur.org/u/mafernandez)\ **Post date:** [September 19, 2019, 2:06pm UTC](https://forum.mothur.org/t/making-otus-without-distance-matrix/20111/5 "2019-09-19T14:06:58Z")
This is the command line I normally use: `dist.seqs(fasta=current, cutoff=0.03, output=column)` Alternatively, I use output=lt Is anything missing? ---
**Author:** ![pschloss](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.mothur.org/pschloss/32/4_2.png) [@pschloss](https://forum.mothur.org/u/pschloss)\ **Post date:** [September 19, 2019, 3:32pm UTC](https://forum.mothur.org/t/making-otus-without-distance-matrix/20111/6 "2019-09-19T15:32:54Z")
You don’t want the lt output format. The problem is likely data quality. If you aren’t able to get fully overlapping reads, then you’ll have to use the phylotype approach shown in the MiSeq SOP. Pat ---
**Author:** ![mafernandez](https://avatars.discourse-cdn.com/v4/letter/m/45deac/32.png) [@mafernandez](https://forum.mothur.org/u/mafernandez)\ **Post date:** [September 19, 2019, 4:16pm UTC](https://forum.mothur.org/t/making-otus-without-distance-matrix/20111/7 "2019-09-19T16:16:56Z")
Thanks, I will try that also. However, the latest analyses I have made are a merge of 2 dataset (one with 26 samples and the other with 9) that worked OK separately. The problem is when you put them together. Both datasets are 16S from bacteria (same primers, same lab) from the same sampling points and from similar environments (soil and rocks). Would it be a problem of the number of samples? About the ‘pre.cluster’ question, would this step be assimilated as a OTU clustering at different similarity levels as I proposed before? ---
**Author:** ![Kendra](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.mothur.org/kendra/32/112_2.png) [@Kendra](https://forum.mothur.org/u/Kendra)\ **Post date:** [September 19, 2019, 8:10pm UTC](https://forum.mothur.org/t/making-otus-without-distance-matrix/20111/8 "2019-09-19T20:10:22Z")
Do you have 2x250 miseq sequences or are they shorter? how much data per sample? have you tried the opticlust method? ---
**Author:** ![system](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.mothur.org/system/32/2_2.png) [@system](https://forum.mothur.org/u/system)\ **Post date:** [September 29, 2019, 8:10pm UTC](https://forum.mothur.org/t/making-otus-without-distance-matrix/20111/9 "2019-09-29T20:10:23Z")
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.