Finding a range of numbers of a file in another file using awk

Question

I have lots of files like this:

3 
10 
23
.
.
.
720
810
980

And a much bigger file like this:

2 0.004
4 0.003
6 0.034
. 
.
.
99         


        

Answer 1

Since your smaller files are sorted you can pull out the first row and the last row to get the min and max. Then you just need go through the bigfile with an awk script to compute the mean.

So for each smallfile small you would run the script

awk -v start=$(head -n 1 small) -v end=$(tail -n 1 small) -f script bigfile

Where script can be something simple like

BEGIN {
    sum = 0;
    count = 0;
    range_start = -1;
    range_end = -1;
}
{
    irow = int($1)
    ival = $2 + 0.0
    if (irow >= start && end >= irow) {
            if (range_start == -1) {
                range_start = NR;
            }
            sum = sum + ival;
            count++;
        }
    else if (irow > end) {
            if (range_end == -1) {
                range_end = NR - 1;
            }
        }
}
END {
    print "start =", range_start, "end =", range_end, "mean =", sum / count
}