get.communitytype() bug: TERM environment variable not set

mothur bugs
1

Hi All,

I am trying to run get.communitytype() with three failed outcomes:

  1. Running from R on a macbook (system() command used to call mothur in batch mode): a done.temp (0 bytes) and .dmm.mix.fit (29 bytes) are created, both do not change in size. R console gives “TERM environment variable not set”. Mothur processes keep working, but within 30 minutes, system runs out of memory.

  2. Running from terminal on a macbook: same as from within R but without the “TERM environment variable not set”. Terminal also starts a table for calculated K values etc but never populates it. Mothur processes keep working, but within 30 minutes, system runs out of memory.

  3. Running on the Michigan State HPCC (single node, non-MPI mothur): a done.temp (0 bytes) and .dmm.mix.fit (29 bytes) are created, both do not change in size. Job runs out of time after hours on a 64 GB node. I tried 32, 20 and 1 processors, results are the same. The job error log says “TERM environment variable not set”, just like R console did on my local machine.

I’m working with about a third of a 250bp paired-end MiSeq run. What am I doing wrong and what is this TERM environment error that’s ruining my day :frowning:


Paul
2

update: okay, there must be more than one issue at work:

the great: success! get.communitytype() works using the mothur-produced shared file and processors=1

the good: TERM environment error goes away using the mothur-produced shared file and the groups parameter to subset the data i.e. select the groups I want analyzed. Before, I would write my own “shared” file using R. Question: is there anything special about the mothur-produced shared file that is not obvious? I try to make my shared file look just like the mothur one. Otu names are not consecutive, but otherwise, all columns are named the same and entries are tab-separated.

the bad: now I get a Segmentation fault: 11 and sometimes “singular matrix” when using the mothur-produced shared file and processors=4 or processors=8

3

Hmmm… How many processors do you actually have? And this is linux, right? Is it a VM?

4

Hi Pat and All,

After a few days of trial and error, I learned that the function executes without error if it is given a mothur-made .shared file and not more processors than the machine actually has. Thanks for all your tips! Although, why I can’t seem to make a mothur shared file manually so far remains a mystery. If I find a solution, I’ll post it! Thanks!

Paul

5

Hi Paul,
Thanks for updating this thread and I am glad you were able to get things working. I would like to look a little deeper into the issue. Would you mind sending your shared file to mothur.bugs@gmail.com? I’d like to see why mothur had such an issue with it and perhaps make her more flexible.
Thanks,
Sarah

6

Thanks, Sarah. I am sending the shared file.

Also, to answer Pat’s previous post, the mostly resolved issue works on two setups:

  1. quad-core macbook with Mac OS X 10.9 compiled 1.33.3
  2. Michigan State cluster with a multi-threaded, non-MPI 1.33.2 module. Single node, 20 cores and 64Gb RAM.

Paul

7

Since this error occurs regularly, and this post is at the top of Google searches, I thought I should post the fix for “TERM environment variable not set”. This error happens because of the way Mothur (and lots of other software) does I/O, if you put “$ export TERM=linux” in the qsub/lsf/etc batch file it should fix it.

charlie