Can Hive recursively descend into subdirectories without partitions or editing hive-site.xml?

Question

I have some web server logs that I\'d like to query with Hive. The directory structure, in HDFS, looks like this:

/data/access/web1/2014/09
/data/access/web1         


        

Answer 1

settings from ozw1z5rd's post worked on Hortonworks

alter table .... set blproperties (
    "hive.input.dir.recursive" = "TRUE",
    "hive.mapred.supports.subdirectories" = "TRUE",
    "hive.supports.subdirectories" = "TRUE",
    "mapred.input.dir.recursive" = "TRUE");    

Answer 2

If you are using ambari the set the following properties to hive advanced config inside custom hive-site.xml.

**- SET hive.input.dir.recursive=TRUE

SET hive.mapred.supports.subdirectories=TRUE

SET hive.supports.subdirectories=TRUE

SET mapred.input.dir.recursive=TRUE**

And then restart the affected services. This will read all the data recursively.

Answer 3

These are not tables properties.

TBLPROPERTIES ("hive.input.dir.recursive" = "TRUE", 
    "hive.mapred.supports.subdirectories" = "TRUE",
    "hive.supports.subdirectories" = "TRUE", 
    "mapred.input.dir.recursive" = "TRUE");

A) add

  <property>
    <name>mapred.input.dir.recursive</name>
    <value>true</value>
  </property>

  <property>
    <name>hive.mapred.supports.subdirectories</name>
    <value>true</value>
  </property>

in hive-site.xml

Answer 4

Using Hive in HDInsight, I set the following properties before I create my external table in the Hive query and it works for me.

SET hive.mapred.supports.subdirectories=TRUE;
SET mapred.input.dir.recursive=TRUE;