random-sample

I have a 10^7 lines file, in which I want to choose 1/100 of lines randomly from the file. This is the AWK code I have, but it slurps all the file content before hand. My PC memory cannot handle such slurps. Is there other approach to do it? awk 'BEGIN{srand()} !/^$/{ a[c++]=$0} END { for ( i=1;i<=c ;i++ ) { num=int(rand() * c) if ( a[num] ) { print a[num] delete a[num] d++ } if ( d == c/100 ) break } }' file cadrian if you have that many lines, are you sure you want exactly 1% or a statistical estimate would be enough? In that second case, just randomize at 1% at each line... awk 'BEGIN

I am using the following code which generates random number between 0 to Totalfriends, I would like to get the random numbers but they should not be repeated. Any idea how? This is the code I am using FB.getLoginStatus(function(response) { var profilePicsDiv = document.getElementById('profile_pics'); FB.api({ method: 'friends.get' }, function(result) { // var result =resultF.data; // console.log(result); var user_ids="" ; var totalFriends = result.length; // console.log(totalFriends); var numFriends = result ? Math.min(25, result.length) : 0; // console.log(numFriends); if (numFriends > 0) {

I'm looking for an efficient way to select rows from a data table such that I have one representative row for each unique value in a particular column. Let me propose a simple example: require(data.table) y = c('a','b','c','d','e','f','g','h') x = sample(2:10,8,replace = TRUE) z = rep(y,x) dt = as.data.table( z ) my objective is to subset data table dt by sampling one row for each letter a-h in column z. OP provided only a single column in the example. Assuming that there are multiple columns in the original dataset, we group by 'z', sample 1 row from the sequence of rows per group, get the

This question asks about getting a random(ish) sample of records on SQL Server and the answer was to use TABLESAMPLE . Is there an equivalent in Oracle 10? If there isn't, is there a standard way to get a random sample of results from a query set? For example how can one get 1,000 random rows from a query that will return millions normally? SELECT * FROM ( SELECT * FROM mytable ORDER BY dbms_random.value ) WHERE rownum <= 1000 grokster The SAMPLE clause will give you a random sample percentage of all rows in a table. For example, here we obtain 25% of the rows: SELECT * FROM emp SAMPLE(25) The

What is the efficient(probably vectorized with Matlab terminology) way to generate random number of zeros and ones with a specific proportion? Specially with Numpy? As my case is special for 1/3 , my code is: import numpy as np a=np.mod(np.multiply(np.random.randomintegers(0,2,size)),3) But is there any built-in function that could handle this more effeciently at least for the situation of K/N where K and N are natural numbers? Yet another approach, using np.random.choice : >>> np.random.choice([0, 1], size=(10,), p=[1./3, 2./3]) array([0, 1, 1, 1, 1, 0, 0, 0, 0, 0]) A simple way to do this