Why do my Perl tests fail with use encoding 'utf8'?

Question

I\'m puzzled with this test script:

#!perl

use strict;
use warnings;
use encoding \'utf8\';
use Test::More \'no_plan\';

ok(\'áá\' =~ m/á/, \'ok direct matc         


        

Answer 1

Do not use the encoding pragma. It’s broken. (Juerd Waalboer gave a great talk where he mentioned this at YAPC::EU 2k8.)

It does at least two things at once that do not belong together:

1. It specifies an encoding for your source file.
2. It specifies an encoding for your file input/output.

And to add injury to insult it also does #1 in a broken fashion: it reinterprets \xNN sequences as being undecoded octets as opposed to treating them like codepoints, and decodes them, preventing you from being able to express characters outside the encoding you specified and making your source code mean different things depending on the encoding. That’s just astonishingly wrong.

Write your source code in ASCII or UTF-8 only. In the latter case, the utf8 pragma is the correct thing to use. If you don’t want to use UTF-8, but you do want to include non-ASCII charcters, escape or decode them explicitly.

And use I/O layers explicitly or set them using the open pragma to have I/O automatically transcoded properly.

Answer 2

It works fine on my computer (on perl 5.10). Maybe you should try replacing that use encoding 'utf8' with use utf8.

What version of perl are you using? I think older versions had bugs with UTF-8 in regexps.

Answer 3

The Test::More documentation contains a fix for this issue, which I just found today (and this entry shows higher in the googles).

utf8 / "Wide character in print"

If you use utf8 or other non-ASCII characters with Test::More you might get a "Wide character in print" warning. Using binmode STDOUT, ":utf8" will not fix it. Test::Builder (which powers Test::More) duplicates STDOUT and STDERR. So any changes to them, including changing their output disciplines, will not be seem by Test::More. The work around is to change the filehandles used by Test::Builder directly.

my $builder = Test::More->builder;
binmode $builder->output,         ":utf8";
binmode $builder->failure_output, ":utf8";
binmode $builder->todo_output,    ":utf8";

I added this bit of boilerplate to my testing code and it works a charm.