Why am I getting an “Out of memory” error with Perl's XML::Simple? [closed]

Answer 1

Without being able to see the code or a representative example, I am reduced to guessing. My bet is that the XML file is very large and you are running out of memory trying to store the data structure. Switch to a stream style XML parser like XML::Twig.

Usually, huge XML files will be be a set of records and you wind up just looping over the data structure you build. A better, faster way of doing this is to process the file as you go:

#!/usr/bin/perl

use strict;
use warnings;

use XML::Twig;

sub process_record {
    my $rec = shift;
    print "@{$rec}{qw/field1 field2 field3/}\n";
}

my $t = XML::Twig->new(
    twig_handlers => {
        record => sub {
            my ($t, $elt) = @_;

            my %record = map {
                $_ => $elt->first_child($_)->text
            } qw/field1 field2 field3/;

            process_record(\%record);

            #get rid of any memory that is being held
            $t->purge;
        },
    }
);

$t->parse(\*DATA);

__DATA__

    
        1
        a
        foo
    
    
        2
        b
        bar
    
    
        3
        c
        baz