Why am I getting an “Out of memory” error with Perl's XML::Simple? [closed]

Answer 1

Without being able to see the code or a representative example, I am reduced to guessing. My bet is that the XML file is very large and you are running out of memory trying to store the data structure. Switch to a stream style XML parser like XML::Twig.

Usually, huge XML files will be be a set of records and you wind up just looping over the data structure you build. A better, faster way of doing this is to process the file as you go:

#!/usr/bin/perl

use strict;
use warnings;

use XML::Twig;

sub process_record {
    my $rec = shift;
    print "@{$rec}{qw/field1 field2 field3/}\n";
}

my $t = XML::Twig->new(
    twig_handlers => {
        record => sub {
            my ($t, $elt) = @_;

            my %record = map {
                $_ => $elt->first_child($_)->text
            } qw/field1 field2 field3/;

            process_record(\%record);

            #get rid of any memory that is being held
            $t->purge;
        },
    }
);

$t->parse(\*DATA);

__DATA__
<root>
    <record>
        <field1>1</field1>
        <field2>a</field2>
        <field3>foo</field3>
    </record>
    <record>
        <field1>2</field1>
        <field2>b</field2>
        <field3>bar</field3>
    </record>
    <record>
        <field1>3</field1>
        <field2>c</field2>
        <field3>baz</field3>
    </record>
</root>

Answer 2

Whenever XML::Simple isn't doing exactly what you want, it's time to use something better. It's designed to work with smallish hashes, not solve all of your XML needs. Besides that, you'll have to tell us a lot more about your situation before we can even guess at the problem.

When you run into a problem, try to reproduce it in a small test case. That might be as simple as a program to read the file and access it. Remember, you're the primary person responsible for solving your problems. :)

Answer 3

To give you the leg up here is your updated example in XML::Twig

use strict;
use warnings;
use XML::Twig;

XML::Twig->new(
    twig_handlers => {
        'c1/rrcConnectionSetupComplete' => \&my_stuff,
    }
)->parsefile( 'uL-DCCH-Message.xml' );

sub my_stuff {
    my ($twig, $elt) = @_;

    my $rrc_trans_identifier = $elt->field( 'rrc-TransactionIdentifier' );
    print "rrc_trans_id :: $rrc_trans_identifier\n";

    my $twiglet = $elt->first_child('criticalExtensions')
                      ->first_child('c1')
                      ->first_child('rrcConnectionSetupComplete-r8');


    my $selected_plmn_id = $twiglet->first_child_text('selectedPLMN-Identity');
    print "plmn identity :: $selected_plmn_id\n";    

    my $rrc_dedicated_info_nas = $twiglet->first_child_text('dedicatedInfoNAS');
    print "dedicated info nas :: $rrc_dedicated_info_nas\n";

    $twig->purge;
}

This outputs exactly what you were after:

rrc_trans_id :: 2
plmn identity ::  1 
dedicated info nas ::  07410109014290112345671000028020000f0 

Important

• Above works for the example you provided but there are a few different ways you could achieve the same result in XML::Twig depending on what your exact requirements are.

• Please don't assign me with the accepted answer for this question! Chas & eugene were first on the scene with the correct answer to your problem and brian followed up with the ubiquitous answer.

/I3az/

Answer 4

For large XML files you could use a SAX parser, like XML::SAX. See perldoc XML::SAX::Intro.
XML::Simple is better suited for parsing small files, e.g. configs.