error message for cluster.split

hi all,

I’m having an issue where the temp files being split from my fasta file can’t be read:

mothur >
cluster.split(fasta=pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta, count=pingo.trim.contigs.unique.good.pick.filter.unique.precluster.count_table, taxonomy=pingo.trim.contigs.unique.good.pick.filter.unique.precluster.nr_v123.wang.taxonomy, splitmethod=classify, taxlevel=5, cutoff=0.03, processors=6)

Using 6 processors.
Using splitmethod fasta.
Splitting the file…
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.507.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.508.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.509.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.510.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.511.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.512.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.513.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.514.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.515.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.516.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.517.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.518.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.519.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.520.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.521.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.522.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.523.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.524.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.525.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.526.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.527.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.528.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.529.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.530.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.531.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.532.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.533.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.534.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.535.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.536.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.537.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.538.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.539.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.540.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.541.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.542.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.543.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.544.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.545.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.546.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.547.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.548.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.549.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.550.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.551.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.552.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.553.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.554.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.555.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.556.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.557.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.558.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.559.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.560.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.561.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.562.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.563.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.564.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.565.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.566.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.567.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.568.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.569.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.570.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.571.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.572.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.573.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.574.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.575.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.576.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.577.temp
[ERROR]: Could not open pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.578.temp
/******************************************/
Running command: dist.seqs(fasta=pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.0.temp, processors=6, cutoff=0.035)

Using 6 processors.
/******************************************/

Output File Names:
pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.0.dist

It took 623 seconds to calculate the distances for 23000 sequences.
/******************************************/
Running command: dist.seqs(fasta=pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.1.temp, processors=6, cutoff=0.035)

Using 6 processors.
/******************************************/

Output File Names:
pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.1.dist

It took 891 seconds to calculate the distances for 28198 sequences.
/******************************************/
Running command: dist.seqs(fasta=pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.2.temp, processors=6, cutoff=0.035)

Using 6 processors.
/******************************************/

Output File Names:
pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.2.dist

It took 124 seconds to calculate the distances for 10511 sequences.
/******************************************/
Running command: dist.seqs(fasta=pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.3.temp, processors=6, cutoff=0.035)

Using 6 processors.
/******************************************/

Output File Names:
pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.3.dist

It took 152 seconds to calculate the distances for 10886 sequences.
/******************************************/
Running command: dist.seqs(fasta=pingo.trim.contigs.unique.good.pick.filter.unique.precluster.fasta.4.temp, processors=6, cutoff=0.035)

Using 6 processors.
/******************************************/



moral of the story it's spitting out .dist files as expected, but it can't read or open a bunch of the files that it split itself.

Can you delete the temp files and try again? Also, can you double check how much hard drive space you have?

Pat

i’ve done it a few times and it always ends this way.

as for hard drive space, not a whole lot, maybe 25 gigs. do you think that’s why?

Thanks for reporting this bug. The operating system has a limit on the number of open files allowed. The way mothur divided the file caused you to hit the limit and then caused a crash. The issue will be fixed in our next release. If you send your input files to mothur.bugs@gmail.com, I will help you move forward with your analysis.