问题
The latest release mahout 0.9 is only built on hadoop 1.x. (mvn clean install) How can I compile mahout for hadoop 2.0.x?
Because When I was running the commands:
hadoop jar mahout-examples-0.9-SNAPSHOT-job.jar org.apache.mahout.cf.taste.hadoop.item.RecommenderJob -s SIMILARITY_COOCCURENCE -i test -o result
I always got the error message
IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected.
Thanks!
回答1:
To compile Mahout to work with 2.x since it isn't released in a package that is compatible with Hadoop 2.x:
mvn clean install -Dhadoop2 -Dhadoop2.version=2.2.0 -DskipTests=true
If you want to confirm after the build that the correct dependencies were brought in, run the following from the root of the project:
find . -name hadoop*.jar
The artifacts generated by the above command differ from what's in the 0.9 release so you'll need to update more than just the version number (there seems to be a new mahout-mrlegacy jar).
That being said, if someone can point me to an official artifact that runs with 2.x (even if it's a SNAPSHOT), I'd appreciate it.
回答2:
Did you try changing the version of hadoop in the pom.xml?
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version> 0.23.9</version>
(Please check the version of hadoop. As I understand, 0.23 became 2.0)
Another thing to try is to run the RecommenderJob directly like this:
bin/mahout recommenditembased \
--input INPUT --output OUTPUT \
--tempDir TEMP --similarityClassname SIMILARITY_LOGLIKELIHOOD
To run this you have to be in the root of your mahout folder, and you need to have the following environment variables set:
export HADOOP_HOME=/path/to/hadoop/home
export HADOOP_COMMONS_HOME=/path/to/commons/home
export JAVA_HOME=/path/to/java/home
export PATH="$PATH:$HADOOP_HOME/bin"
回答3:
About an hour ago Mahout has officially added support to Hadoop 2.x on the master branch (see MAHOUT-1329)
Checkout the code here https://github.com/apache/mahout and recompile using:
mvn clean package -Dhadoop2.version=2.2.0
Try and see if that works.
来源：https://stackoverflow.com/questions/18767843/how-can-i-compile-using-mahout-for-hadoop-2-0