Could not find resource path for Web UI: org/apache/spark/ui/static While creating a Spark application

问题

I am creating a java API using spark-cassandra-connector.When it is running on eclipse it is working fine. Then i created a runnable jar file and executed it through the command line , its giving me following error.

15/02/17 14:56:41 INFO spark.HttpServer: Starting HTTP Server
    15/02/17 14:56:42 INFO server.Server: jetty-8.y.z-SNAPSHOT
    15/02/17 14:56:42 INFO server.AbstractConnector: Started SocketConnector@0.0.0.0:58625
    Exception in thread "main" java.lang.reflect.InvocationTargetException
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
            at java.lang.reflect.Method.invoke(Unknown Source)
            at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoa
    der.java:58)
    Caused by: java.lang.Exception: Could not find resource path for Web UI: org/apache/spark/ui/static    
    at org.apache.spark.ui.JettyUtils$.createStaticHandler(JettyUtils.scala:
    133)
            at org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:70)
            at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:60)
            at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:42)
            at org.apache.spark.SparkContext.<init>(SparkContext.scala:222)
            at org.apache.spark.SparkContext.<init>(SparkContext.scala:97)
            at com.netspective.watchtower.bigdata.utils.Config.getSparkContext(Confi
    g.java:32)
            at com.netspective.watchtower.bigdata.main.Main.main(Main.java:17)
            ... 5 more

The only difference between the log files comparing to eclipse log is regarding the version of jetty-server. In eclipse it identifiesthe jetty server version as the 8.1.14 v232 .But above its identifying as 8.y.z SNAPSHOT. Is this because of jetty server or any other reason? I am totally stuck here.

My sparkContext configuration is follows

  conf.setAppName("Java API demo");
        conf.setMaster("local[*]");

回答1:

Spark uses "getResource" to read files that are packaged with it. Here it tries to read some static files for the Web UI, which apparently does not work if the files are packaged into the jar. Instead, they must be extracted into the jar, which Eclipse offers as the first option:

来源：https://stackoverflow.com/questions/28559018/could-not-find-resource-path-for-web-ui-org-apache-spark-ui-static-while-creati

标签

java

eclipse

apache-spark

jetty-8