Perl Merge file

Question

I have 3 or multiple files I need to merge, the data looks like this..

file 1
0334.45656
0334.45678
0335.67899
file 2
0334.89765
0335.12346
0335.56789
file 3         


        

Answer 1

I wouldn't normally suggest this, but unix utilties should be able to handle this just fine.

1. cat the 3 files together.
2. use sort to sort the merged file.

However, using perl, could just do the following:

#!/usr/bin/perl

use strict;
use warnings;

my @data;
push @data, $_ while (<>);

# Because the numbers are all equal length, alpha sort will work here
print for sort @data;

However, as we've discussed, it's possible that the files will be extremely large. Therefore it will be more efficient both in memory and speed if you're able to take advantage of the fact that all the files are already sorted.

The following solution therefore streams the files, pulling out the next one in order each loop of the while:

#!/usr/bin/perl

# Could name this catsort.pl

use strict;
use warnings;
use autodie;

# Initialize File handles
my @fhs = map {open my $fh, '<', $_; $fh} @ARGV;

# First Line of each file
my @data = map {scalar <$_>} @fhs;

# Loop while a next line exists
while (@data) {
    # Pull out the next entry.
    my $index = (sort {$data[$a] cmp $data[$b]} (0..$#data))[0];

    print $data[$index];

    # Fill In next Data at index.
    if (! defined($data[$index] = readline $fhs[$index])) {
        # End of that File
        splice @fhs, $index, 1;
        splice @data, $index, 1;
    }
}