Efficient operations of big non-sparse matrices in Matlab

问题

I need to operate in big 3-dim non-sparse matrices in Matlab. Using pure vectorization gives a high computation time. So, I have tried to split the operations into 10 blocks and then parse the results. I got surprised when I saw the the pure vectorization does not scale very well with the data size as presented in the following figure.

I include an example of the two approaches.

% Parameters:
M = 1e6;  N = 50;  L = 4;  K = 10;

% Method 1: Pure vectorization
mat1 = randi(L,[M,N,L]);
mat2 = repmat(permute(1:L,[3 1 2]),M,N);
result1 = nnz(mat1>mat2)./(M+N+L);

% Method 2: Split computations
result2 = 0;
for ii=1:K
    mat1 = randi(L,[M/K,N,L]);
    mat2 = repmat(permute(1:L,[3 1 2]),M/K,N);
    result2 = result2 + nnz(mat1>mat2);
end
result2 = result2/(M+N+L);

Hence, I wonder if there is any other approach that makes big matrix operations in Matlab more efficient. I know it is a quite broad question, but I will take the risk :)

---

Edit:

Using the implementation of @Shai

% Method 3
mat3 = randi(L,[M,N,L]);
result3 = nnz(bsxfun( @gt, mat3, permute( 1:L, [3 1 2] ) ))./(M+N+L);

The times are:

回答1:

Why repmat and not bsxfun?

result = nnz(bsxfun( @gt, mat1, permute( 1:L, [3 1 2] ) ))./(M+N+L);

It seems like you are using up your RAM and the OS starts to allocate room in swap for the very large matrics. Memory swapping is always a very time consuming operation and it gets worse as the amount of memory you require increases.
I believe you are witnessing thrashing.

来源：https://stackoverflow.com/questions/22383813/efficient-operations-of-big-non-sparse-matrices-in-matlab