Hi,
Thought I would start another thread on this, rather than munge up the previous one on the commands board. I’ve observed that when running cluster.split() with large=F, then the memory for the distance matrix is read entirely on a single box, rather than spreading across nodes. This seems like non-ideal behavior given that compute nodes are often limited in memory (ours have 8GB). Is this the design or a bug?
Below is the usage I’m talking about with 50 nodes (pid, mem, cpu, time, cmd). The box with 5G of used memory is the master MPI node. It’s also interesting that the threads that don’t seem to be doing anything in terms of memory usage are all consuming a single processor. The column matrix in question is 61GB.
21329 25m 99 46:00.1 mothurMPI
9049 25m 101.1 46:01.8 mothurMPI
9050 25m 99.1 46:01.4 mothurMPI
411 25m 101 46:03.1 mothurMPI
24634 25m 99.1 46:04.3 mothurMPI
29341 25m 101.2 46:04.8 mothurMPI
21338 25m 101.2 46:07.2 mothurMPI
5380 25m 101 46:06.6 mothurMPI
29716 25m 101.2 46:09.0 mothurMPI
29715 25m 99.2 46:10.1 mothurMPI
4435 25m 101.2 46:11.5 mothurMPI
1035 25m 100.7 46:11.2 mothurMPI
6783 25m 101.2 46:14.1 mothurMPI
16922 25m 101.1 46:15.6 mothurMPI
9452 25m 99.2 46:16.8 mothurMPI
15526 25m 99.2 46:18.4 mothurMPI
15527 25m 99.2 46:18.2 mothurMPI
16556 25m 101.1 46:19.6 mothurMPI
16557 25m 99.2 46:19.4 mothurMPI
31458 25m 99.1 46:21.0 mothurMPI
17575 25m 99.1 46:22.3 mothurMPI
19930 25m 100.9 46:23.6 mothurMPI
19929 25m 99 46:23.8 mothurMPI
24870 25m 99.5 46:24.8 mothurMPI
24869 25m 97.7 46:24.9 mothurMPI
24475 25m 99.2 46:27.0 mothurMPI
5635 25m 100 46:27.6 mothurMPI
5636 25m 100 46:27.6 mothurMPI
3873 25m 101.1 46:29.0 mothurMPI
18771 25m 101 46:31.4 mothurMPI
18772 25m 101 46:31.0 mothurMPI
6608 25m 101.1 46:32.5 mothurMPI
6607 25m 99.1 46:32.9 mothurMPI
25178 25m 101.2 46:34.5 mothurMPI
25179 25m 101.2 46:33.9 mothurMPI
22468 25m 101.1 46:33.8 mothurMPI
974 25m 101.1 46:37.8 mothurMPI
975 25m 99.1 46:37.6 mothurMPI
13796 25m 99.2 46:38.6 mothurMPI
26788 25m 99.3 46:40.6 mothurMPI
27733 25m 101.1 46:42.2 mothurMPI
27734 25m 101.1 46:42.0 mothurMPI
9786 25m 99.2 46:44.3 mothurMPI
16272 25m 99.1 46:45.5 mothurMPI
6997 5.0g 99.5 46:02.5 mothurMPI
6998 25m 99.5 46:46.1 mothurMPI
25913 25m 99.8 46:47.7 mothurMPI
25914 25m 99.8 46:47.7 mothurMPI
13675 25m 99.8 46:49.0 mothurMPI
13676 25m 99.8 46:49.1 mothurMPI
Thanks,
Chris