sqoop 从mysql 导入数据到hbase

环境:

软件	版本	备注
Ubuntu	19.10
sqoop	1.4.7
mysql	8.0.20-0ubuntu0.19.10.1 (Ubuntu)
hbase	2.2.4	必须启动
hadoop	3.1.2	必须启动
hive	3.0.0	之所以和hive有关系是因为需要在.bashrc中设置HCAT_HOME
accumulo	2.0.0	需要配合sqoop在.bashrc中设置ACCUMULO_HOMT

数据导入目标:

mysql数据------------->Hbase

##############################################################################

准备MYSQL数据集:

mysql> create database sqoop_hbase;
mysql> use sqoop_hbase;
mysql> CREATE TABLE book(
-> id INT(4) PRIMARY KEY NOT NULL AUTO_INCREMENT,
-> NAME VARCHAR(255) NOT NULL,
-> price VARCHAR(255) NOT NULL);
插入数据集
mysql> INSERT INTO book(NAME, price) VALUES('Lie Sporting', '30');
mysql> INSERT INTO book (NAME, price) VALUES('Pride & Prejudice', '70');
mysql> INSERT INTO book (NAME, price) VALUES('Fall of Giants', '50');

##############################################################################

需要.bashrc中设置好ACCUMULO_HOME和HCAT_HOME

############################下面是准备迁移##################################################

终端输入命令:

sqoop import --connect jdbc:mysql://Desktop:3306/sqoop_hbase \
--username appleyuchi \
--password appleyuchi \
--table book \
--columns "id,name,price" \
--column-family "info" \
--hbase-create-table \
--hbase-row-key "id" \
--hbase-table "hbase_book" \
--num-mappers 1 \
--split-by id

##############################################################################

hbase shell

scan 'hbase_book'

最终结果如下:

#-----------------------------------------------附录-报错解决方案--------------------------------------------------------------------------------------

报错	$SQOOP/lib需要补充jar
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver	mysql-connector-java-8.0.20.jar
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.HBaseAdmin	hbase-client-1.2.0.jar
`Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/protobuf/generated/MasterProtos$MasterService$BlockingInterface`	hbase-protocol-1.2.0.jar
java.lang.NoSuchMethodError: org.apache.hadoop.hbase.util.Hash.hash([BII)I	hbase-common-1.2.0.jar
java.lang.ClassNotFoundException: com.yammer.metrics.core.Gauge	metrics-core-3.2.6.jar
Caused by: java.lang.ClassNotFoundException: org.apache.htrace.Trace	htrace-core-3.1.0-incubating.jar