问题
I have written different code to produce different permutations of ones and minus ones. they work for matrixes with small dimensions:
for example:
S=[-1 -1 1 1 1 1 1 1];
P=unique(perms(S),'rows');
produces:
-1 -1 1 1 1 1 1 1
-1 1 -1 1 1 1 1 1
-1 1 1 -1 1 1 1 1
-1 1 1 1 -1 1 1 1
-1 1 1 1 1 -1 1 1
-1 1 1 1 1 1 -1 1
-1 1 1 1 1 1 1 -1
1 -1 -1 1 1 1 1 1
1 -1 1 -1 1 1 1 1
1 -1 1 1 -1 1 1 1
1 -1 1 1 1 -1 1 1
1 -1 1 1 1 1 -1 1
1 -1 1 1 1 1 1 -1
1 1 -1 -1 1 1 1 1
1 1 -1 1 -1 1 1 1
1 1 -1 1 1 -1 1 1
1 1 -1 1 1 1 -1 1
1 1 -1 1 1 1 1 -1
1 1 1 -1 -1 1 1 1
1 1 1 -1 1 -1 1 1
1 1 1 -1 1 1 -1 1
1 1 1 -1 1 1 1 -1
1 1 1 1 -1 -1 1 1
1 1 1 1 -1 1 -1 1
1 1 1 1 -1 1 1 -1
1 1 1 1 1 -1 -1 1
1 1 1 1 1 -1 1 -1
1 1 1 1 1 1 -1 -1
or
indices = nchoosek(1:41, 6);
N = size(indices, 1);
S = ones(N, 41);
S(sub2ind([N 41], [1:N 1:N 1:N 1:N 1:N 1:N].', indices(:))) = -1;
can produce a matrix of 4496388_by_41 of all the permutations of 6 minus one(-1) and 35 one(1).
these codes work for smaller dimensions but they don't work for the matrixs with larger dimensions.
my goal is to produce all permutations of 20 minus one(-1) and 21 one(1) this matrix has 269128937220 rows and 41 columns. but the following codes don't work:
indices = nchoosek(1:41, 20);
N = size(indices, 1);
S = ones(N, 41);
S(sub2ind([N 41], [1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N 1:N].', indices(:))) = -1;
or
S=[-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1];
P=unique(perms(S),'rows');
I do a simple calculation on each permutation(each row of this matrix). if I could write each row of this matrix with for loops and then do the calculation on that row, I would be able to keep the best result and in this situation I wouldn't have to keep all these data in the memory and I wouldn't get out of memory errors from matlab.
if you know how to produce a matrix of all the permutations of 20 minus one(-1) and 21 one(1) with for loops or any other way to store them in my computer please help.
thanks in advance
回答1:
I'm not an expert in Matlab so I can't speak for all of the resources available, however, I know that your task is feasible on a standard laptop without any fancy high performance services such as https://aws.amazon.com/hpc/.
I have authored a package in R called RcppAlgos that is capable of completing this task comfortably in a few hours. Here is the code:
options(scipen = 999)
library(parallel)
library(RcppAlgos)
## WARNING Don't run this unless you have a few hours on your hand
## break up into even intervals of one million
firstPart <- mclapply(seq(1, 269128000000, 10^6), function(x) {
temp <- permuteGeneral(c(1L,-1L), freqs = c(21,20), lower = x, upper = x + 999999)
## your analysis here
x
}, mc.cores = 8)
## get the last few results and complete analysis
lastPart <- permuteGeneral(c(1L, -1L), freqs = c(21, 20),
lower = 269128000000, upper = 269128937220)
## analysis for last part goes here
And to give you a demonstration of the efficiency of this setup, we will demonstrate how fast the first one billion results are completed.
system.time(mclapply(seq(1, 10^9, 10^6), function(x) {
temp <- permuteGeneral(c(1L, -1L), freqs = c(21, 20), lower = x, upper = x + 999999)
## your analysis here
x
}, mc.cores = 8))
user system elapsed
121.158 64.057 27.182
Under 30 seconds for 1000000000 results!!!!!!!
So, this will not take over 3000 days as @CrisLuengo calculated but rather a conservative estimate of 30 seconds per billion gives :
(269128937220 / 1000000000 / 60) * 30 ~= 134.5645 minutes
I should also note that with the setup above you are only using 1251.2 Mb at a time, so your memory will not explode.
testSize <- object.size(permuteGeneral(c(1L,-1L), freqs = c(21,20), upper = 1e6))
print(testSize, units = "Mb")
156.4 Mb ## per core
All results were obtained on a MacBook Pro 2.8GHz quad core (with 4 virtual cores.. 8 total).
Edit:
As @CrisLuengo points out, the above only measures generating that many permutations and does not factor in the time taken for analysis of each computation. After some more clarification and a new question, we have that answer now... about 2.5 days!!!
来源：https://stackoverflow.com/questions/51564122/how-to-produce-every-permutation-of-positioning-20-values-of-1-in-a-1-by-41-vec