thread shared perl

Question

i wrote a code and i need to make it multithreaded. Evething works, but every loop repeats 4 times:

use LWP::UserAgent;
use HTTP::Cookies;
use threads;
use threa         


        

Answer 1

The problem is that you never remove from @groups, so all threads do all jobs in @groups.

Here's one solution.

use threads;
use Thread::Queue 3.01 qw( );

my $NUM_WORKERS = 4;

sub worker {
   my ($url) = @_;
   ... download the page ...
}

my $q = Thread::Queue->new();
for (1..$NUM_WORKERS) {
   async {
      while (my $url = $q->dequeue()) {
         worker($url);
      }
   };
}

$q->enqueue($_) for loadf('groups.txt');
$q->end();
$_->join() for threads->list;

Answer 2

Why do you need to make it threaded? perl does much better using forks in most cases.

That said, your code starts 4 threads, each of which processes everything in @groups. It sounds like that's not what you want to do. If you want @groups to be a queue of work to do, take a look at Thread::Queue (or Parallel::ForkManager).