How can I screen-scrape output from telnet in Perl?

Question

I can setup a telnet connection in Perl no problems, and have just discovered Curses, and am wondering if I can use the two together to scrape the output from the telnet ses

Answer 1

You're looking for Term::VT102, which emulates a VT102 terminal (converting the terminal control characters back into a virtual screen state). There's an example showing how to use it with Net::Telnet in VT102/examples/telnet-usage.pl (the examples directory is inside the VT102 directory for some reason).

It's been about 7 years since I used this (the system I was automating switched to a web-based interface), but it used to work.

Answer 2

You probably want something like Expect

use strict;
use warnings;

use Expect;

my $exp = Expect->spawn("telnet google.com 80");

$exp->expect(15, #timeout
        [
                qr/^Escape character.*$/,
                sub {
                        $exp->send("GET / HTTP/1.0\n\n");
                        exp_continue;
                }
        ]
);

Answer 3

I would vote also for the Expect answer. I had to do something similar from a gui'ish application. The trick (albeit tedious) to get around the control characters was to strip all the misc characters from the returned strings. It kind of depends on how messy the screen scrape ends up being.

Here is my function from that script as an example:

# Trim out the curses crap
sub trim {
    my @out = @_;
    for (@out) {
        s/\x1b7//g;
        s/\x1b8//g;
        s/\x1b//g;   # remove escapes
        s/\W\w\W//g;
        s/\[\d\d\;\d\dH//g;  # 
        s/\[\?25h//g;
        s/\[\?25l//g;
        s/\[\dm//g;
        s/qq//g;
        s/Recall//g;
        s/\357//g;
        s/[^0-9:a-zA-Z-\s,\"]/ /g;
        s/\s+/ /g;    # Extra spaces

    }
    return wantarray ? @out : $out[0];
}

Answer 4

If you are interacting purely with plain-text commands and responses, you can use Expect to script that, otherwise, you can use Term::VT102, which lets you screen scrape (read specific parts of the screen, send text, handle events on scrolling, cursor movement, screen content changes, and others) applications using VT102 escape sequences for screen control (e.g., an application using the curses library).

Answer 5

Or you could use the script command for this.

From the Solaris man-page:

DESCRIPTION

The script utility makes a record of everything printed on your screen. The record is written to filename. If no file name is given, the record is saved in the file typescript...

The script command forks and creates a sub-shell, according to the value of $SHELL, and records the text from this session. The script ends when the forked shell exits or when Control-d is typed.

Answer 6

Curses does the opposite. It is a C library for optimising screen updates from a program writing to a terminal, originally designed to be used over a slow serial connection. It has no ability to scrape a layout from a sequence of control characters.

A better bet would be a terminal emulator that has an API with the ability to do this type of screen scraping. Off the top of my head I'm not sure if any Open-source terminal emulators do this, but there are certainly commercial ones available that can.