IPC::Open3 Fails Running Under Apache

Question

I have a module that uses IPC::Open3 (or IPC::Open2, both exhibit this problem) to call an external binary (bogofilter in this case) and feed it some input via the child-input filehandle, then reads the result from the child-output handle. The code works fine when run in most environments. However, the main use of this module is in a web service that runs under Apache 2.2.6. And under that environment, I get the error:

Cannot fdopen STDOUT: Invalid argument

This only happens when the code runs under Apache. Previously, the code constructed a horribly complex command, which included a here-document for the input, and ran it with back-ticks. THAT worked, but was very slow and prone to breaking in unique and perplexing ways. I would hate to have to revert to the old version, but I cannot crack this.

Answer 1

Could it be because mod_perl 2 closes STDOUT? I just discovered this and posted about it:

http://marc.info/?l=apache-modperl&m=126296015910250&w=2

I think it's a nasty bug, but no one seems to care about it thus far. Post a follow up on the mod_perl list if your problem is related and you want it to get attention.

Jon

Answer 2

Bogofilter returns different exit codes for spam/nonspam.

You can "fix" this by redirecting stdout to /dev/null

system("bogofilter < $input > /dev/null") >> 8;

Will return 0 for spam, 1 for nonspam, 2 for unknown (the >> 8 is because perl helpfully corrects the exit code, this fixes the damage).

Note: the lack of an environment may also prevent bogofilter from finding its wordlist, so pass that in explicitly as well:

system("bogofilter -d /path/to/.bogofilter/ < $input > /dev/null") >> 8;

(where /path/to/.bogofilter contains the wordlist.db)

You can't retrieve the actual rating that bogofilter gave that way, but it does get you something.

Answer 3

If your code is only going to be run on Linux/Unix systems it is easy to write an open3 replacement that does not fail because STDOUT is not a real file handle:

sub my_open3 {
    # untested!
    pipe my($inr), my($inw) or die;
    pipe my($outr), my($outw) or die;
    pipe my($errr), my($errw) or die;
    my $pid = fork;
    unless ($pid) {
        defined $pid or die;
        POSIX::dup2($inr, 0);
        POSIX::dup2($outw, 1);
        POSIX::dup2($errw, 2);
        exec @_;
        POSIX::_exit(1);
    }
    return ($inw, $outr, $errr);
}

my ($in, $out, $err) = my_open3('ls /etc/');

Answer 4

Caveat Emptor: I am not a perl wizard.

As @JonathanSwartz suggested, I believe the issue is that apache2 mod_perl closes STDIN and STDOUT. That shouldn't be relevant to what IPC::Open3 is doing, but it has a bug in it, described here.

In summary (this is the part I'm not super clear on), open3 tries to match the child processes STDIN/OUT/ERR to your process, or duplicate it if that was what is requested. Due to some undocumented ways that open('>&=X') works, it generally works fine, except in the case where STDIN/OUT/ERR are closed.

Another link that gets deep into the details.

One solution is to fix IPC::Open3, as described in both of those links. The other, which worked for me, is to temporarily open STDIN/OUT in your mod_perl code and then close it afterwards:

my ($save_stdin,$save_stdout);
open $save_stdin, '>&STDIN';
open $save_stdout, '>&STDOUT';
open STDIN, '>&=0';
open STDOUT, '>&=1';

#make your normal IPC::Open3::open3 call here

close(STDIN);
close(STDOUT);
open STDIN, '>&', $save_stdin;
open STDOUT, '>&', $save_stdout;

Also, I noticed a bunch of complaints around the net about IPC::Run3 suffering from the same problems, so if anyone runs into the same issue, I suspect the same solution would work.