Parsing unsorted data from large fixed width text

Question

I am mostly a Matlab user and a Perl n00b. This is my first Perl script.

I have a large fixed width data file that I would like to process into a binary file with a

Answer 1

It seems to me that embedded spaces can only occur in the last field. That makes using split ' ' feasible for this problem.

I am assuming you are not interested in the header. In addition, I am assuming you want a vector for each parameter and are not interested in timestamps.

To use data file names specified on the command line or piped through standard input, replace with <>.

#!/usr/bin/env perl

use strict; use warnings;

my %data;

$_ =  until /^-+/; # skip header

while (my $line = ) {
    $line =~ s/\s+\z//;
    last unless $line =~ /\S/;

    my (undef, $param, undef, $value) = split ' ', $line, 4;
    push @{ $data{ $param } }, $value;
}

use Data::Dumper;
print Dumper \%data;

__DATA__
Param1   filter = ALL_VALUES
Param2   filter = ALL_VALUES
Param3   filter = ALL_VALUES

Time                     Name     Ty  Value
---------- ---------------------- --- ------------
1          Param1                 UI  5
2          Param3                 TXT Some Text 1
3          Param1                 UI  10
4          Param2                 D   2.1234
5          Param1                 UI  15
6          Param2                 D   3.1234
7          Param3                 TXT Some Text 2

Output:

$VAR1 = {
          'Param2' => [
                        '2.1234',
                        '3.1234'
                      ],
          'Param1' => [
                        '5',
                        '10',
                        '15'
                      ],
          'Param3' => [
                        'Some Text 1',
                        'Some Text 2'
                      ]
        };