Finding a range of numbers of a file in another file using awk

Question

I have lots of files like this:

3 
10 
23
.
.
.
720
810
980

And a much bigger file like this:

2 0.004
4 0.003
6 0.034
. 
.
.
99         


        

Answer 1

You can try below:

for r in *;  do
    awk -v r=$r -F' ' \
    'NR==1{b=$2;v=$4;next}{if(r >= b && r <= $2){m=(v+$4)/2; print m; exit}; b=$2;v=$4}' bigfile.txt
done

Explanation:

First pass it saves column 2 & 4 into temp variables. For all other passes it checks if filename r is between the begin range (previous coluimn 2) and end range (current column 2). It then works out the mean and prints the result.

Answer 2

Since your smaller files are sorted you can pull out the first row and the last row to get the min and max. Then you just need go through the bigfile with an awk script to compute the mean.

So for each smallfile small you would run the script

awk -v start=$(head -n 1 small) -v end=$(tail -n 1 small) -f script bigfile

Where script can be something simple like

BEGIN {
    sum = 0;
    count = 0;
    range_start = -1;
    range_end = -1;
}
{
    irow = int($1)
    ival = $2 + 0.0
    if (irow >= start && end >= irow) {
            if (range_start == -1) {
                range_start = NR;
            }
            sum = sum + ival;
            count++;
        }
    else if (irow > end) {
            if (range_end == -1) {
                range_end = NR - 1;
            }
        }
}
END {
    print "start =", range_start, "end =", range_end, "mean =", sum / count
}