mothur

Error with cluster.split - cannot open opti.mcc.list

I was running the cluster.split command on a large data set and it started returning errors saying it couldn’t open the opti.mcc.list files. Not sure what’s going on here, but hoping someone might have some insight.

Command: cluster.split(fasta=AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta,count=AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.denovo.vsearch.pick.pick.pick.count_table,taxonomy=AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.nr_v128.wang.pick.pick.taxonomy, splitmethod=classify, taxlevel=4, cutoff=0.03,processors=32)

Part of log file output:
It took 709838 seconds to split the distance file.
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.9.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.75.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.93.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.116.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.89.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.218.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.242.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.247.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.217.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.143.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.336.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.394.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.254.dist
AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.444.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.444.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.413.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.453.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.366.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.312.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.463.dist

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
0 0 0 3 0 0 0 0 1 0 0 0

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.455.dist

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
0 12 0 3 0 1 0 0.8 1 0.8 0 0

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.368.dist
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.455.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
18 27 0 0 1 1 1 1 1 1 1 1

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.345.dist
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.368.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
0 6 0 15 0 1 0 0.285714 1 0.285714 0 0

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.473.dist

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
0 3 0 3 0 1 0 0.5 1 0.5 0 0

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.292.dist
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.345.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
48 3517 2 3 0.941176 0.999432 0.96 0.999148 0.96 0.998599 0.949833 0.950495

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.122.dist

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
0 0 0 3 0 0 0 0 1 0 0 0

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.320.dist
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.473.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
36 47 0 8 0.818182 1 1 0.854545 1 0.912088 0.836166 0.9

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.376.dist

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
4 17 0 0 1 1 1 1 1 1 1 1

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.254.dist

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
3 75 0 0 1 1 1 1 1 1 1 1

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.392.dist
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.292.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
0 6 0 15 0 1 0 0.285714 1 0.285714 0 0

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.422.dist
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.122.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
153 1598 11 8 0.950311 0.993163 0.932927 0.995019 0.932927 0.989266 0.935678 0.941538

[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.320.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
19 538 4 0 1 0.99262 0.826087 1 0.826087 0.99287 0.905533 0.904762

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.360.dist
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.376.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
113 696 9 43 0.724359 0.987234 0.92623 0.941813 0.92623 0.939605 0.785935 0.81295

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.370.dist
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.254.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
20 721 0 0 1 1 1 1 1 1 1 1

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.142.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.394.dist
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.370.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
396 3500 12 8 0.980198 0.996583 0.970588 0.997719 0.970588 0.994893 0.972535 0.975369

**** Exceeded maximum allowed command errors, quitting ****
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.392.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
0 40 0 15 0 1 0 0.727273 1 0.727273 0 0

**** Exceeded maximum allowed command errors, quitting ****
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.422.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
33 186 2 10 0.767442 0.989362 0.942857 0.94898 0.942857 0.948052 0.82155 0.846154

**** Exceeded maximum allowed command errors, quitting ****
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.360.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
51 202 0 0 1 1 1 1 1 1 1 1

**** Exceeded maximum allowed command errors, quitting ****
[ERROR]: Could not open AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.394.opti_mcc.list

tp tn fp fn sensitivity specificity ppv npv fdr accuracy mcc f1score
52 297 1 1 0.981132 0.996644 0.981132 0.996644 0.981132 0.994302 0.977776 0.981132

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.390.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.339.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.420.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.456.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.449.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.375.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.408.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.468.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.451.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.385.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.459.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.424.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.348.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.474.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.369.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.465.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.425.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.421.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.426.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.464.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.450.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.158.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.416.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.447.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.442.dist

Clustering AmphStability.trim.contigs.good.unique.good.filter.unique.precluster.pick.pick.pick.fasta.172.dist




Detected 14 [ERROR] messages, please review.




srun: error: UV00000395-P001: task 0: Exited with exit code 1
Mothur Run Failed with exit code 1 !!!

No need to use cluster.split if you are using opti distances. You may also be exceeding your RAM, try this instead

dist.seqs(fasta=current, countends=F, cutoff= 0.03, processors=16)

cluster(column=current, count=current, method=opti)