LWP::UserAgent Insists on verifying hostname

Question

The following script works returns a comprehensive headers on a host running libwww-perl-5.836 but not on the host using libwww-perl-6.30.0. In that c

Answer 1

You have turned off verifying the hostname; you have not turned off verifying the certificate.

verify_hostname controls whether "LWP will for secure protocol schemes ensure it connects to servers that have a valid certificate matching the expected hostname" (my emphasis). Setting that to 0 allows you to connect to a server that has a valid certificate but not issued for the host / hostname that you are trying to reach.

To turn off checking that the certificate is valid (issued by a trusted CA), you want:

use IO::Socket::SSL;

my $ua = LWP::UserAgent->new(
   ssl_opts => {
      verify_hostname => 0,
      SSL_verify_mode => IO::Socket::SSL::SSL_VERIFY_NONE,
   },
);

Note that turning off either of these options is a bad idea if you are transmitting any sensitive information or expect to trust the data returned. With either of these turned off, you are losing the benefits of SSL and are vulnerable to various man-in-the-middle attacks.