Error in pig while loading data

Question

I am using ubuntu 12.02 32bit and have installed hadoop2.2.0 and pig 0.12 successfully. Hadoop runs properly on my system.

However,

Answer 1

I've resolved it in another way.

I got the same problem on CDH4.4 & Pig 0.11.0 when my Pig script was invoking a UDF from my java project which was compiled using Maven. I've visited /usr/lib/pig/conf/build.properties file. Verified the versions mentioned against hadoop-core, hadoop-common, hadoop-mapreduce properties. Ensured that all these artifacts with same versions are included as dependencies in my java project's POM.xml file.

( Infact hadoop-mapreduce has 6 artifact Ids as per http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_31.html. I've included all of them in the dependencies list).

After building my project's jar file with these POM settings, Pig script was able to invoke UDF without any problem.

Answer 2

Apache Pig 0.12.0 expects an older version of Hadoop by default. You must recompile Pig for Hadoop 2.2.0 and replace two jars with new pig-0.12.1-SNAPSHOT.jar and pig-0.12.1-SNAPSHOT-withouthadoop.jar.

For recompilation unpack the pig archive, go to the directory "pig-0.12.0" and just run:

ant clean jar-all -Dhadoopversion=23

Answer 3

Just caught the same exception, trying to run script with my UDF on Hadoop 2x.

In latest release 0.14 and current trunk 0.15, ant target jar-all is missing. So if you are building from sources, you can use

jar-h12 Create pig for both Hadoop 1 and Hadoop 2

or

ant jar -Dhadoopversion=23

or simply download compatible version from Maven Central.

In my case it was pig-0.14.0-h2.jar

Answer 4

just building with command "ant clean jar-all -Dhadoopversion=23" is not enough if you are using maven dependencies in your project. You will need to install the jar created by this in your local maven repo or use this dependency (notice "classifier" tag for hadoop2) in your pom.xml

<dependency>
<groupId>org.apache.pig</groupId>
<artifactId>pig</artifactId>
<classifier>h2</classifier>
<version>0.13.0</version>
</dependency>

Answer 5

This is your clue ERROR org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl - Error while trying to run jobs. java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected. The interface "jobcontrol" is looking for a certain class and doesn't find it.

Answer 6

This problem also arises with any hadoop version above 2.2. An easier way is to install Pig version 0.15. They have solved this problem in this version.