What's hive-site.xml including in $SPARK_HOME looks like?

Question

I am a beginner at hive, something happened (can not find table) when I start spark job and read data from hive. I don\'t set hive-site.xml in $SPARK_HOME/conf ?

submit

Answer 1

Just copy the hive-site.xml to conf directory and it will work.

Answer 2

On your hive distribution you have a template file that must convert into your own site file.

https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration#AdminManualConfiguration-hive-site.xmlandhive-default.xml.template

So first of all you must create your own hive-site.xml file by copying the hive-default.xml.template and then you can use it from spark.

If you don't want to use the default file, you can use any of the previous configurations shown on the previous answers.

Answer 3

I believe it might depend on the distribution you're using. I encountered this problem recently, and this fixed the issue for me. I'm using HDP 2.3.2 so my copy of hive-site.xml in the Spark conf folder only contains this:

<configuration>

  <property>
  <name>hive.metastore.uris</name>
  <value>thrift://host.xxx.com:9083</value>
  </property>

</configuration>

Answer 4

hive-site.xml look like the following .

<configuration>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://localhost/metastore_db?createDatabaseIfNotExist=true</value>
        <description>metadata is stored in a MySQL server</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
        <description>MySQL JDBC driver class</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>USER</value>
        <description>user name for connecting to mysql server </description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>PASSWORD</value>
        <description>password for connecting to mysql server </description>
    </property>
</configuration>