Hadoop: strange ClassNotFoundException

南楼画角 提交于 2020-01-07 04:07:29

问题


I am getting a classnotfound exception. The class which is claimed to be not found does not exist, but the class name is set as the path to the list of input files for my map reduce jobs.

INFO  server Running: /usr/lib/hadoop/bin/hadoop --config /var/run/cloudera-scm-agent/process/155-hue/JOBSUBD/hadoop-conf jar tmp.jar /user/hduser/datasets/ /user/hduser/tmp/job_20/ mongodb://slave15/db_8.job_20

Exception in thread "main" java.lang.ClassNotFoundException: /user/hduser/datasets/

at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(Runjar.java:190)

As we can see, the /user/hduser/datasets/ is the path to the input files. Why am I getting this error as ClassNotFoundException? Why does it see it as a class?


I found my own error. I have a package structure. I need to specify my package information

/usr/lib/hadoop/bin/hadoop 
    --config /var/run/cloudera-scm-agent/process/155-hue/JOBSUBD/hadoop-conf 
    jar tmp.jar org.myorg.tmp /user/hduser/datasets/ 
    /user/hduser/tmp/job_20/ mongodb://slave15/db_8.job_20

In my tool, there is no option for giving the package as argument to Java. So I need to have no packaging. But then I am having the following error since the argument before this input file path is missing.

My classes are directly in the tmp.jar in its root. I mean no org.myorg etc...

SOLUTION:

jar cmf [manifest_file] [jar_name.jar] -C [folder_of_classes] [path_for_jar_file]

it will merge the content of the manifest_file with the generated manifest file in the jar archive. Include the following line in the manifest_file Main-Class: [Name_Of_Class]


回答1:


Whether or not there's a package hierarchy (and if there isn't, you've done Something Wrong) you still need to give it the name of the class containing main.

For example, the docs have an example where the main class is org.myorg.WordCount. Even if WordCount was in the default package, it should be specified if the jar file doesn't include a main class in the manifest:

bin/hadoop jar /usr/joe/wordcount.jar WordCount /usr/joe/wordcount/input /usr/joe/wordcount/output

I'd assume you could also specify the main class in the manifest as with any jar; the class argument is shown as optional in those same docs.



来源:https://stackoverflow.com/questions/10976846/hadoop-strange-classnotfoundexception

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!