How to load a flat file(not delimited file) into HBase?

老子叫甜甜 提交于 2019-12-12 05:29:09

问题


I am new to hbase and I have a flat file(not delimited file) that I would like to load into a single hbase table.

Here is a preview of a row in my file:

0107E07201512310015071C11100747012015123100

I know fo example that from position 1 to 7 it's an id and from position 7 to 15 it's a date....

The problem is how to build a schema that correspond to my file or if there is a way to convert it to a delimited file or read such file using jaql because I'm working with Infosphere BigInsights.

Any help would be greatly appreciated.

Thanks in advance.


回答1:


Create a Hive table using RegExSerDe

CREATE EXTERNAL TABLE testtable ((col1 STRING, col2 STRING, col3 STRING)
ROW FORMAT SERDE ‘org.apache.hadoop.hive.contrib.serde2.RegexSerDe’
WITH SERDEPROPERTIES (“input.regex” = “(.{5})(.{6})(.{3}).*” )
LOCATION ‘<hdfs-file-location>’;

You can create hive table pointing to HBase Here are the instructions http://hortonworks.com/blog/hbase-via-hive-part-1/

You can use insert overwrite table to load the data from hive table to HBase-table https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-SELECTSandFILTERS




回答2:


You can write a SerDe to deserialize into Hive and use Hive to export to HBase.



来源:https://stackoverflow.com/questions/36703803/how-to-load-a-flat-filenot-delimited-file-into-hbase

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!