how to run a class from java jar file on Hadoop?

拟墨画扇 提交于 2020-01-15 10:06:45

问题


I have a jar file exported from Eclipse (Runnable JAR>>Copy required libraries into a sub folder).

In java if you set the main class in the manifest.xml you can run:

java -jar MyTest.jar arguments

if you want to run another main class in the jar file or if you didn't set a main class in the manifest.xml you can run:

java -cp MyTest.jar MyOtherMainClass arguments

In Hadoop if main class is set in manifest.xml you can run:

hadoop jar MyTest.jar arguments

If you type:

hadoop jar MyTest.jar MyOtherMainClass arguments

It will consider MyOtherMainClass as argument (not as a class to run) in the "args" array of the original main class of jar.

Now if you want to run another main class in the jar file what will you type?

I expect something similar to:

hadoop java -cp MyTest.jar MyOtherMainClass arguments

but that gives:

Error: Could not find or load main class java

Notice: if I remove "hadoop" from "hadoop java -cp MyTest.jar MyOtherMainClass arguments" it will launch normally


回答1:


The problem comes from Eclipse forcing you to set the main class in the jar file and hence preventing you to run the class that you want. All you have to do is remove the main class from the manifest.xml file of the jar file and run:

hadoop jar MyTest.jar MyOtherMainClass arguments

Take a look here: http://www.roman10.net/2012/07/26/specify-the-main-class-in-a-jar-file-for-hadoop/

I typed the same text in case the url deleted:

Hadoop support execution of jar file. For an executable jar file in normal java execution, one can specify the main class in the command line, as covered in my previous post: switch between main classes in a jar file.

However, the rules are a bit different for executable jar file running with hadoop. Basically the following rules hold (I tested on Hadoop 1.0.3),

If a jar file contains a main class specified in its manifest file, hadoop will take the main class even if the command specify another main class. This is different from normal java execution where we can specify a main class to overwrite the one in the manifest file. If a jar file does not contain a main class in manifest file, hadoop allows us to specify the main class. At eclipse, when one export a project as runnable jar file, it always ask for a main class at Launch configuration.

The main class selected will be put in the manifest file. Below is the content of the META-INF/MANIFEST.MF file in my helloworld project where the main class is set to HelloWorld.

Manifest-Version: 1.0 Class-Path: . Main-Class: hello.HelloWorld One can browse the jar file using a file extractor, open the manifest file using a file editor, and simply delete the last line to remove the main class configuration, and save the changes to the jar file when prompted. This will create a runnable jar file without main class.

The modified jar file can then be used in Hadoop with user supplied main class configuration, as shown in the sample command below,

$ hadoop jar hello.jar hello.HelloWorld



来源:https://stackoverflow.com/questions/43017899/how-to-run-a-class-from-java-jar-file-on-hadoop

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!