lwp-useragent

LWP::UserAgent Insists on verifying hostname

对着背影说爱祢 提交于 2020-05-11 04:56:25
问题 The following script works returns a comprehensive headers on a host running libwww-perl-5.836 but not on the host using libwww-perl-6.30.0 . In that case, the script displays the following: 500 Can't connect to backend.mutegroup.org:443 (certificate verify failed) Content-Type: text/plain Client-Date: Mon, 28 Jul 2014 21:09:28 GMT Client-Warning: Internal response Can't connect to backend.mutegroup.org:443 (certificate verify failed) LWP::Protocol::https::Socket: SSL connect attempt failed

How can I use Perl's LWP::UserAgent to fetch the same URL with different query strings?

白昼怎懂夜的黑 提交于 2019-12-25 18:34:38
问题 I have a running LWP::UserAgent that should be applied on following URL: http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=5503 This runs with many many similar targets see the following endings: html?show_school=5503 html?show_school=9002 html?show_school=5512 I want to do this with use LWP::UserAgent: for my $i (0..10000) { $ua->get(' [here the URL should be applied] ', id => 21, extern_uid => $i); # process reply } In any case, using a loop like this for that kind

Timeout for LWP request

戏子无情 提交于 2019-12-25 04:33:05
问题 I'm creating a user agent using LWP::UserAgent. I want to timeout $ua->request() . how can I do that? my $ua = LWP::UserAgent->new(); my $request = HTTP::Request->new(DELETE => $url); $req->authorization_basic($user, $pwd); my $response = $ua->request($req); I know LWP has time out, but according to the documentation The requests is aborted if no activity on the connection to the server is observed for timeout seconds. This means that the time it takes for the complete transaction and the

Why can't LWP::UserAgent get this site entirely?

非 Y 不嫁゛ 提交于 2019-12-23 17:45:09
问题 It outputs only a few lines from the beginning. #!/usr/bin/perl use strict; use warnings; use LWP::UserAgent; my $ua = LWP::UserAgent->new; my $response = $ua->get('http://www.eurogamer.net/articles/df-hardware-wii-u-graphics-power-finally-revealed'); print $response->decoded_content; 回答1: I ran the following modification: my $response = $ua->get( 'http://www.eurogamer.net/articles/df-hardware-wii-u-graphics-power-finally-revealed' ); say $response->headers->as_string; And saw this: Cache

LWP::UserAgent Can't Post with TLS1.1

青春壹個敷衍的年華 提交于 2019-12-22 07:08:02
问题 Getting 500 handshaker error:443 over https. The host service I am sending XML to does not support TLS 1.2, they do support 1.0 and 1.1. Currently using LWP 6.03 on CentOS 6. Using the code below they claim I am still sending using TLS1.2 use LWP::UserAgent; $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0,SSL_version => 'SSLv23:!TLSv12' }); $req = HTTP::Request->new(GET => 'https://secure-host-server'); $res = $ua->request($req); if ($res->is_success) { print $res->content; }

How to enforce a definite timeout in perl?

你。 提交于 2019-12-18 06:48:35
问题 I am using LWP to download content from web pages, and I would like to limit the amount of time it waits for a page. my $ua = LWP::UserAgent->new; $ua->timeout(10); $ua->env_proxy; $response = $ua->get("http://XML File"); $content = $response->decoded_content; The problem is that the server will occasionally deadlock (we're trying to figure out why) and the request will never succeed. Since the server thinks it is live, it keeps the socket connection open thus LWP::UserAgent's timeout value

perl retrieving page details after mechanize::POST

此生再无相见时 提交于 2019-12-12 18:41:19
问题 I am trying to gather data from a website. Some anti-patterns make looking finding the right form objects difficult but I have this solved. I am using a post method to get around some javascript acting as a wrapper to submit the form. My problem seems to be in getting the results from the mechanize->post method. Here's a shortened version of my code. use strict; use warnings; use HTML::Tree; use LWP::Simple; use WWW::Mechanize; use HTTP::Request::Common; use Data::Dumper; $| = 1; my $site_url

Mojo::UserAgent get() with userdefined callback

自作多情 提交于 2019-12-12 17:23:59
问题 Is there a way to write something like this with Mojo::UserAgent ,having the possibility to set equivalent options ( Range , :content_file , :content_cb , size_hint ). #!/usr/bin/env perl use warnings; use 5.12.0; use LWP::UserAgent; use File::Basename; my $url = 'ftp://ftp.vim.org/pub/vim/runtime/spell/en.utf-8.spl'; my $file = basename $url; my $ua = LWP::UserAgent->new(); my $bytes = 0; open my $fh, '>>:raw', $file or die $!; my $res = $ua->get( $url, 'Range' => "bytes=$bytes-", ':content

LWP::UserAgent HTTP Basic Authentication

空扰寡人 提交于 2019-12-12 08:29:22
问题 I tried to run this perl5 program: #!/usr/bin/env perl use strict; use warnings; use LWP; my $ua = LWP::UserAgent->new('Mozilla'); $ua->credentials("test.server.com:39272", "realm-name", 'user_name', 'some_pass'); my $res = $ua->get('http://test.server.com:39272/'); print $res->content; On other hand I have HTTP::Daemon: #!/usr/bin/env perl use strict; use warnings; use HTTP::Daemon; my $hd = HTTP::Daemon->new or die; print "Contact URL: ", $hd->url, "\n"; while (my $hc = $hd->accept) { while

LWP Get Large File Download Headers Missing

你。 提交于 2019-12-11 21:33:47
问题 This post is follow on work related to LWP GET large file download. That post was regarding an error from LWP when trying to pass arguments in the header incorrectly. Now I am posting the changes I made and how I am trying to debug the approach. This discussion should be very informative for those interested in POST vs GET header formation, and what the server receives while using the CGI package. It is not information easily found on the net. Here is my client code snip: my $bytes_received =