Reducing memory usage in an extended Mathematica session

Question

I\'m doing some rather long computations, which can easily span a few days. In the course of these computations, sometimes Mathematica will run out of memory. To this end, I\'ve

Answer 1

Since you are using Module extensively, I think you may be interested in knowing this bug with non-deleting temporary Module variables.

Example (non-deleting unlinked temporary variables with their definitions):

In[1]:= $HistoryLength=0;
a[b_]:=Module[{c,d},d:=9;d/;b===1];
Length@Names[$Context<>"*"]

Out[3]= 6

In[4]:= lst=Table[a[1],{1000}];
Length@Names[$Context<>"*"]

Out[5]= 1007

In[6]:= lst=.
Length@Names[$Context<>"*"]

Out[7]= 1007

In[8]:= Definition@d$999

Out[8]= Attributes[d$999]={Temporary}

d$999:=9

Note that in the above code I set $HistoryLength = 0; to stress this buggy behavior of Module. If you do not do this, temporary variables can still be linked from history variables (In and Out) and will not be removed with their definitions due to this reason in more broad set of cases (it is not a bug but a feature, as Leonid mentioned).

UPDATE: Just for the record. There is another old bug with non-deleting unreferenced Module variables after Part assignments to them in v.5.2 which is not completely fixed even in version 7.0.1:

In[1]:= $HistoryLength=0;$Version
Module[{L=Array[0&,10^7]},L[[#]]++&/@Range[100];];
Names["L$*"]
ByteCount@Symbol@#&/@Names["L$*"]
Out[1]= 7.0 for Microsoft Windows (32-bit) (February 18, 2009)
Out[3]= {L$111}
Out[4]= {40000084}