How can we handle large matrices in matlab(larger than 10000x10000)

问题

In my program I am faced with some matrices that are larger than 10000x10000. I cannot transpose or inverse them, how can this problem be overcome?

??? Error using ==> ctranspose
Out of memory. Type HELP MEMORY for your options.
Error in ==> programname1 at 70
    B = cell2mat(C(:,:,s))'; 
Out of memory. Type HELP MEMORY for your options.
Example 1: Run the MEMORY command on a 32-bit Windows system:


    >> memory
    Maximum possible array:             677 MB (7.101e+008 bytes) *
    Memory available for all arrays:   1602 MB (1.680e+009 bytes) **
    Memory used by MATLAB:              327 MB (3.425e+008 bytes)
    Physical Memory (RAM):             3327 MB (3.489e+009 bytes)

    *  Limited by contiguous virtual address space available.
    ** Limited by virtual address space available.

Example 2: Run the MEMORY command on a 64-bit Windows system:

    >> memory
    Maximum possible array:               4577 MB (4.800e+009 bytes) *
    Memory available for all arrays:      4577 MB (4.800e+009 bytes) *
    Memory used by MATLAB:                 330 MB (3.458e+008 bytes)
    Physical Memory (RAM):                3503 MB (3.674e+009 bytes)

==============================================================================

 memory
% Maximum possible array:            1603 MB (1.681e+009 bytes) *
% Memory available for all arrays:   2237 MB (2.346e+009 bytes) **
% Memory used by MATLAB:              469 MB (4.917e+008 bytes)
% Physical Memory (RAM):             3002 MB (3.148e+009 bytes)


I have used sparse for C. 

B = cell2mat(C);
clear C       %#  to reduce the allocated RAM
P=B\b;

Name         Size                  Bytes  Class     Attributes     

  B         5697x5697            584165092  double    sparse, complex
  C         1899x1899            858213576  cell                     
  b         5697x1                   91152  double    complex        

==============================================================================
??? Error using ==> mldivide
Out of memory. Type HELP MEMORY for your options.

Error in ==> programname at 82
    P=B\b; 

==============================================================================

Edit: 27.05.11

Name         Size                  Bytes  Class     Attributes

  C          997x997             131209188  cell   
  B            2991x2991             71568648  single    complex        
  Bdp          2991x2991            143137296  double    complex        
  Bsparse      2991x2991            156948988  double    sparse, complex

  Bdp=double(B);
  Bsparse=sparse(Bdp);

I used single precision, witch gave the same accuracy as in double precision

It's better, Am I right?

回答1:

A few suggestions:

If possible, as @yoda suggested, use sparse matrices
Do you really need the inverse? If you're solving a linear system (Ax=b), you should use MATLAB's backslash operator.
If you really need huge dense matrices, you can harness the memory of several machines using distributed arrays and MATLAB distributed computing server.

回答2:

3GB isn't alot when each matrix you have is 600 MB all by it's self. If you can't make the algorithmic changes, you need 64-bit matlab on a 64-bit OS, with alot more RAM. It's the only way to get alot of memory. Notice that with 3 GB, Matlab only has 2.2 GB, and the largest chunk is 1.5 GB - that's only 2 of your matricies.

回答3:

Matlab has an easy way to handling huge matrices of orders like 1000000*1000000. These matrices are usually sparse matrices and it's not necessary to allocate RAM memory for matrix elements of zero value. So you should just use this command:

A=sparse(1000000,1000000); "Defining a 1000000 by 1000000 matrix of zeroes."

Then you can set the diagonal nonzero elements by commands like "spdiags". See this Link : http://www.mathworks.nl/help/matlab/ref/spdiags.html

Note that you can not use "inv" command to invert matrix A, because "inv" made a normal matrix and uses a lot of space of the RAM.(probably with "out of memory" error)

To solve an equation like A*X=B, you can use X=A\B.

来源：https://stackoverflow.com/questions/6119060/how-can-we-handle-large-matrices-in-matlablarger-than-10000x10000

标签

matlab

large-data-volumes