CDH + phoenix+ zeppelin

久未见 提交于 2019-12-01 17:38:42
  • 内容概述

1.安装及配置Phoenix

2.Phoenix的基本操作

3.使用Phoenix bulkload数据到HBase

4.使用Phoenix从HBase中导出数据到HDFS

  • 测试环境

1.CDH5.11.2

2.RedHat7.2

3.Phoenix4.7.0

  • 前置条件

1.CDH集群正常

2.HBase服务已经安装并正常运行

3.测试csv数据已准备

4.Redhat7中的httpd服务已安装并使用正常

2.在CDH集群中安装Phoenix


1.到Cloudera官网下载Phoenix的Parcel,注意选择与操作系统匹配的版本,因为本次测试使用的是Redhat7,所以选择后缀名为el7的文件。下载地址为:

http://archive.cloudera.com/cloudera-labs/phoenix/parcels/latest/

具体需要下载的三个文件地址为:

http://archive.cloudera.com/cloudera-labs/phoenix/parcels/latest/CLABS_PHOENIX-4.7.0-1.clabs_phoenix1.3.0.p0.000-el7.parcel  http://archive.cloudera.com/cloudera-labs/phoenix/parcels/latest/CLABS_PHOENIX-4.7.0-1.clabs_phoenix1.3.0.p0.000-el7.parcel.sha1  http://archive.cloudera.com/cloudera-labs/phoenix/parcels/latest/manifest.json

2.将下载好的文件发布到httpd服务,可以用浏览器打开页面进行测试。

[ec2-user@ip-172-31-22-86 phoenix]$ pwd  /var/www/html/phoenix  [ec2-user@ip-172-31-22-86 phoenix]$ ll  total 192852  -rw-r--r-- 1 root root        41 Jun 24  2016 CLABS_PHOENIX-4.7.0-1.clabs_phoenix1.3.0.p0.000-el7.parcel.sha1  -rw-r--r-- 1 root root 197466534 Jun 24  2016 CLABS_PHOENIX-4.7.0-1.clabs_phoenix1.3.0.p0.000-el7.parcel  -rw-r--r-- 1 root root      4687 Jun 24  2016 manifest.json  [ec2-user@ip-172-31-22-86 phoenix]$

3.从Cloudera Manager点击“Parcel”进入Parcel管理页面

点击“配置”,输入Phoenix的Parcel包http地址。

点击“保存更改“回到Parcel管理页面,发现CM已发现Phoenix的Parcel。

点击“下载”->“分配”->“激活”

4.回到CM主页,发现HBase服务需要部署客户端配置以及重启

重启HBase服务

安装完成。

3.如何在CDH集群中使用Phoenix

3.1Phoenix的基本操作


进入Phoenix的脚本命令目录

[ec2-user@ip-172-31-22-86 bin]$ cd /opt/cloudera/parcels/CLABS_PHOENIX/bin  [ec2-user@ip-172-31-22-86 bin]$ ll  total 16  -rwxr-xr-x 1 root root 672 Jun 24  2016 phoenix-performance.py  -rwxr-xr-x 1 root root 665 Jun 24  2016 phoenix-psql.py  -rwxr-xr-x 1 root root 668 Jun 24  2016 phoenix-sqlline.py  -rwxr-xr-x 1 root root 674 Jun 24  2016 phoenix-utils.py

使用Phoenix登录HBase

[ec2-user@ip-172-31-22-86 bin]$ ./phoenix-sqlline.py  Zookeeper not specified.   Usage: sqlline.py <zookeeper> <optional_sql_file>   Example:    1. sqlline.py localhost:2181:/hbase    2. sqlline.py localhost:2181:/hbase ../examples/stock_symbol.sql

需要指定Zookeeper

[ec2-user@ip-172-31-22-86 bin]$ ./phoenix-sqlline.py ip-172-31-21-45:2181:/hbase  ...  sqlline version 1.1.8  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> !tables  +------------+--------------+-------------+---------------+----------+------------+--------------------+  | TABLE_CAT  | TABLE_SCHEM  | TABLE_NAME  |  TABLE_TYPE   | REMARKS  | TYPE_NAME  | SELF_REFERENCING_C |  +------------+--------------+-------------+---------------+----------+------------+--------------------+  |            | SYSTEM       | CATALOG     | SYSTEM TABLE  |          |            |                    |  |            | SYSTEM       | FUNCTION    | SYSTEM TABLE  |          |            |                    |  |            | SYSTEM       | SEQUENCE    | SYSTEM TABLE  |          |            |                    |  |            | SYSTEM       | STATS       | SYSTEM TABLE  |          |            |                    |  |            |              | ITEM        | TABLE         |          |            |                    |  +------------+--------------+-------------+---------------+----------+------------+--------------------+  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase>

创建一张测试表

注意:建表必须指定主键。

0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> create table hbase_test  . . . . . . . . . . . . . . . . . . . . . .> (  . . . . . . . . . . . . . . . . . . . . . .>     s1 varchar not null primary key,  . . . . . . . . . . . . . . . . . . . . . .>     s2 varchar,  . . . . . . . . . . . . . . . . . . . . . .>     s3 varchar,  . . . . . . . . . . . . . . . . . . . . . .>     s4 varchar  . . . . . . . . . . . . . . . . . . . . . .> );  No rows affected (1.504 seconds)

在hbase shell中进行检查

插入一行数据。注意:Phoenix中没有insert语法,用upsert代替。参考:http://phoenix.apache.org/language/index.html

0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test values('1','testname','testname1','testname2');  1 row affected (0.088 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> select * from hbase_test;  +-----+-----------+------------+------------+  | S1  |    S2     |     S3     |     S4     |  +-----+-----------+------------+------------+  | 1   | testname  | testname1  | testname2  |  +-----+-----------+------------+------------+  1 row selected (0.049 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase>

在hbase shell中进行检查

删除这行数据,delete测试

0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> delete from hbase_test where s1='1';  1 row affected (0.018 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> select * from hbase_test;  +-----+-----+-----+-----+  | S1  | S2  | S3  | S4  |  +-----+-----+-----+-----+  +-----+-----+-----+-----+  No rows selected (0.045 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase>

在hbase shell中进行检查

更新数据测试,注意Phoenix中没有update语法,用upsert代替。插入多条数据需要执行多条upsert语句,没办法将所有的数据都写到一个“values”后面。

0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test values('1','testname','testname1','testname2');  1 row affected (0.017 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test values('2','testname','testname1','testname2');  1 row affected (0.007 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test values('3','testname','testname1','testname2');  1 row affected (0.008 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> select * from hbase_test;  +-----+-----------+------------+------------+  | S1  |    S2     |     S3     |     S4     |  +-----+-----------+------------+------------+  | 1   | testname  | testname1  | testname2  |  | 2   | testname  | testname1  | testname2  |  | 3   | testname  | testname1  | testname2  |  +-----+-----------+------------+------------+  3 rows selected (0.067 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test values('1','fayson','testname1','testname2');  1 row affected (0.009 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> select * from hbase_test;  +-----+-----------+------------+------------+  | S1  |    S2     |     S3     |     S4     |  +-----+-----------+------------+------------+  | 1   | fayson    | testname1  | testname2  |  | 2   | testname  | testname1  | testname2  |  | 3   | testname  | testname1  | testname2  |  +-----+-----------+------------+------------+  3 rows selected (0.037 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase>

在hbase shell中进行检查

批量更新测试,创建另外一张表hbase_test1,表结构与hbase_test一样,并插入五条,有两条是hbase_test中没有的(主键为4,5),有一条与hbase_test中的数据不一样(主键为1),有两条是完全一样(主键为2,3)。

0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> create table hbase_test1  . . . . . . . . . . . . . . . . . . . . . .> (  . . . . . . . . . . . . . . . . . . . . . .>     s1 varchar not null primary key,  . . . . . . . . . . . . . . . . . . . . . .>     s2 varchar,  . . . . . . . . . . . . . . . . . . . . . .>     s3 varchar,  . . . . . . . . . . . . . . . . . . . . . .>     s4 varchar  . . . . . . . . . . . . . . . . . . . . . .> );  No rows affected (1.268 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase>   0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test1 values('1','fayson','testname1','testname2');  1 row affected (0.031 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test1 values('2','testname','testname1','testname2');  1 row affected (0.006 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test1 values('3','testname','testname1','testname2');  1 row affected (0.005 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test1 values('4','testname','testname1','testname2');  1 row affected (0.005 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test1 values('5','testname','testname1','testname2');  1 row affected (0.007 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> select * from hbase_test1;  +-----+-----------+------------+------------+  | S1  |    S2     |     S3     |     S4     |  +-----+-----------+------------+------------+  | 1   | fayson    | testname1  | testname2  |  | 2   | testname  | testname1  | testname2  |  | 3   | testname  | testname1  | testname2  |  | 4   | testname  | testname1  | testname2  |  | 5   | testname  | testname1  | testname2  |  +-----+-----------+------------+------------+  5 rows selected (0.038 seconds)

批量更新,我们用hbase_test1中的数据去更新hbase_test。

0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> upsert into hbase_test select * from hbase_test1;  5 rows affected (0.03 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> select * from hbase_test;  +-----+-----------+------------+------------+  | S1  |    S2     |     S3     |     S4     |  +-----+-----------+------------+------------+  | 1   | fayson    | testname1  | testname2  |  | 2   | testname  | testname1  | testname2  |  | 3   | testname  | testname1  | testname2  |  | 4   | testname  | testname1  | testname2  |  | 5   | testname  | testname1  | testname2  |  +-----+-----------+------------+------------+  5 rows selected (0.039 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase>

批量更新发现对于已有的数据,如果值不一样,会覆盖,对于相同的数据会保持不变,对于没有的数据会直接作为新的数据插入。

3.2使用Phoenix bulkload数据到HBase


准备需要批量导入的测试数据,这里使用TPC_DS的item表数据。

[ec2-user@ip-172-31-22-86 ~]$ ll item.dat  -rw-r--r-- 1 root root 28855325 Oct  3 10:23 item.dat  [ec2-user@ip-172-31-22-86 ~]$ head -1 item.dat  1|AAAAAAAABAAAAAAA|1997-10-27||Powers will not get influences. Electoral ports should show low, annual chains. Now young visitors may pose now however final pages. Bitterly right children suit increasing, leading el|27.02|23.23|5003002|exportischolar #2|3|pop|5|Music|52|ableanti|N/A|3663peru009490160959|spring|Tsp|Unknown|6|ought|

因为Phoenix的bulkload只能导入csv,所以我们先把该数据的分隔符修改为逗号,并且后缀名改为.csv

[ec2-user@ip-172-31-22-86 ~]$ sed -i 's/|/,/g' item.dat  [ec2-user@ip-172-31-22-86 ~]$ mv item.dat item.csv  [ec2-user@ip-172-31-22-86 ~]$ ll item.csv   -rw-r--r-- 1 ec2-user ec2-user 28855325 Oct  3 10:26 item.csv  [ec2-user@ip-172-31-22-86 ~]$ head -1 item.csv   1,AAAAAAAABAAAAAAA,1997-10-27,,Powers will not get influences. Electoral ports should show low, annual chains. Now young visitors may pose now however final pages. Bitterly right children suit increasing, leading el,27.02,23.23,5003002,exportischolar #2,3,pop,5,Music,52,ableanti,N/A,3663peru009490160959,spring,Tsp,Unknown,6,ought,

上传该文件到HDFS

[ec2-user@ip-172-31-22-86 ~]$ hadoop fs -mkdir /fayson  [ec2-user@ip-172-31-22-86 ~]$ hadoop fs -put item.csv /fayson  [ec2-user@ip-172-31-22-86 ~]$ hadoop fs -ls /fayson  Found 1 items  -rw-r--r--   3 ec2-user supergroup   28855325 2017-10-03 10:28 /fayson/item.csv  [ec2-user@ip-172-31-22-86 ~]$

通过Phoenix创建item表,注意为了方便阅读,只创建了4个字段

0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> create table item  . . . . . . . . . . . . . . . . . . . . . .> (  . . . . . . . . . . . . . . . . . . . . . .>     i_item_sk varchar not null primary key,  . . . . . . . . . . . . . . . . . . . . . .>     i_item_id varchar,  . . . . . . . . . . . . . . . . . . . . . .>     i_rec_start_varchar varchar,  . . . . . . . . . . . . . . . . . . . . . .>     i_rec_end_date varchar  . . . . . . . . . . . . . . . . . . . . . .> );  No rows affected (1.268 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase>

执行bulkload命令导入数据

[ec2-user@ip-172-31-22-86 ~]$ HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH/lib/hbase/hbase-protocol-1.2.0-cdh5.12.1.jar:/opt/cloudera/parcels/CDH/lib/hbase/conf hadoop jar /opt/cloudera/parcels/CLABS_PHOENIX/lib/phoenix/phoenix-4.7.0-clabs-phoenix1.3.0-client.jar org.apache.phoenix.mapreduce.CsvBulkLoadTool -t item -i /fayson/item.csv  17/10/03 10:32:24 INFO util.QueryUtil: Creating connection with the jdbc url: jdbc:phoenix:ip-172-31-21-45.ap-southeast-1.compute.internal,ip-172-31-22-86.ap-southeast-1.compute.internal,ip-172-31-26-102.ap-southeast-1.compute.internal:2181:/hbase;  ...  17/10/03 10:32:24 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=ip-172-31-21-45.ap-southeast-1.compute.internal:2181,ip-172-31-22-86.ap-southeast-1.compute.internal:2181,ip-172-31-26-102.ap-southeast-1.compute.internal:2181 sessionTimeout=60000 watcher=hconnection-0x7a9c0c6b0x0, quorum=ip-172-31-21-45.ap-southeast-1.compute.internal:2181,ip-172-31-22-86.ap-southeast-1.compute.internal:2181,ip-172-31-26-102.ap-southeast-1.compute.internal:2181, baseZNode=/hbase  17/10/03 10:32:24 INFO zookeeper.ClientCnxn: Opening socket connection to server ip-172-31-21-45.ap-southeast-1.compute.internal/172.31.21.45:2181. Will not attempt to authenticate using SASL (unknown error)  ...  17/10/03 10:32:30 INFO mapreduce.Job: Running job: job_1507035313248_0001  17/10/03 10:32:38 INFO mapreduce.Job: Job job_1507035313248_0001 running in uber mode : false  17/10/03 10:32:38 INFO mapreduce.Job:  map 0% reduce 0%  17/10/03 10:32:52 INFO mapreduce.Job:  map 100% reduce 0%  17/10/03 10:33:01 INFO mapreduce.Job:  map 100% reduce 100%  17/10/03 10:33:01 INFO mapreduce.Job: Job job_1507035313248_0001 completed successfully  17/10/03 10:33:01 INFO mapreduce.Job: Counters: 50  ...  17/10/03 10:33:01 INFO mapreduce.AbstractBulkLoadTool: Loading HFiles from /tmp/fef0045b-8a31-4d95-985a-bee08edf2cf9  ...

在Phoenix中查询该表

0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase> select * from item limit 10;  +------------+-------------------+----------------------+-----------------+  | I_ITEM_SK  |     I_ITEM_ID     | I_REC_START_VARCHAR  | I_REC_END_DATE  |  +------------+-------------------+----------------------+-----------------+  | 1          | AAAAAAAABAAAAAAA  | 1997-10-27           |                 |  | 10         | AAAAAAAAKAAAAAAA  | 1997-10-27           | 1999-10-27      |  | 100        | AAAAAAAAEGAAAAAA  | 1997-10-27           | 1999-10-27      |  | 1000       | AAAAAAAAIODAAAAA  | 1997-10-27           | 1999-10-27      |  | 10000      | AAAAAAAAABHCAAAA  | 1997-10-27           | 1999-10-27      |  | 100000     | AAAAAAAAAKGIBAAA  | 1997-10-27           | 1999-10-27      |  | 100001     | AAAAAAAAAKGIBAAA  | 1999-10-28           | 2001-10-26      |  | 100002     | AAAAAAAAAKGIBAAA  | 2001-10-27           |                 |  | 100003     | AAAAAAAADKGIBAAA  | 1997-10-27           |                 |  | 100004     | AAAAAAAAEKGIBAAA  | 1997-10-27           | 2000-10-26      |  +------------+-------------------+----------------------+-----------------+  10 rows selected (0.054 seconds)  0: jdbc:phoenix:ip-172-31-21-45:2181:/hbase>

在hbase shell中查询该表

hbase(main):002:0> scan 'ITEM', LIMIT => 10  ROW                         COLUMN+CELL                                                                    1                          column=0:I_ITEM_ID, timestamp=1507041176470, value=AAAAAAAABAAAAAAA            1                          column=0:I_REC_START_VARCHAR, timestamp=1507041176470, value=1997-10-27        1                          column=0:_0, timestamp=1507041176470, value=                                   10                         column=0:I_ITEM_ID, timestamp=1507041176470, value=AAAAAAAAKAAAAAAA            10                         column=0:I_REC_END_DATE, timestamp=1507041176470, value=1999-10-27             10                         column=0:I_REC_START_VARCHAR, timestamp=1507041176470, value=1997-10-27        10                         column=0:_0, timestamp=1507041176470, value=                                  ...   100004                     column=0:I_REC_START_VARCHAR, timestamp=1507041176470, value=1997-10-27        100004                     column=0:_0, timestamp=1507041176470, value=                                  10 row(s) in 0.2360 seconds

入库条数检查

条数相等,全部入库成功。

3.3使用Phoenix从HBase中导出数据到HDFS


Phoenix还提供了使用MapReduce导出数据到HDFS的功能,以pig的脚本执行。首先准备pig脚本。

[ec2-user@ip-172-31-22-86 ~]$ cat export.pig   REGISTER /opt/cloudera/parcels/CLABS_PHOENIX/lib/phoenix/phoenix-4.7.0-clabs-phoenix1.3.0-client.jar;  rows = load 'hbase://query/SELECT * FROM ITEM' USING org.apache.phoenix.pig.PhoenixHBaseLoader('ip-172-31-21-45:2181');  STORE rows INTO 'fayson1' USING PigStorage(',');
[ec2-user@ip-172-31-22-86 ~]$

执行该脚本

[ec2-user@ip-172-31-22-86 ~]$ pig -x mapreduce export.pig   ...  Counters:  Total records written : 102000  Total bytes written : 4068465  Spillable Memory Manager spill count : 0  Total bags proactively spilled: 0  Total records proactively spilled: 0    Job DAG:  job_1507035313248_0002    2017-10-03 10:45:38,905 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!

导出成功后检查HDFS中的数据

[ec2-user@ip-172-31-22-86 ~]$ hadoop fs -ls /user/ec2-user/fayson1  Found 2 items  -rw-r--r--   3 ec2-user supergroup          0 2017-10-03 10:45 /user/ec2-user/fayson1/_SUCCESS  -rw-r--r--   3 ec2-user supergroup    4068465 2017-10-03 10:45 /user/ec2-user/fayson1/part-m-00000  [ec2-user@ip-172-31-22-86 ~]$ hadoop fs -cat /user/ec2-user/fayson1/part-m-00000 | head -2  1,AAAAAAAABAAAAAAA,1997-10-27,  10,AAAAAAAAKAAAAAAA,1997-10-27,1999-10-27  cat: Unable to write to output stream.  [ec2-user@ip-172-31-22-86 ~]$

检查条数为10200与原始数据一致,全部导出成功。

4.总结


  • 使用Cloudera提供的Phoenix Parcel,可以很方便的安装Phoenix。
  • 使用Phoenix可以对HBase进行建表,删除,更新等操作,都是以大家熟悉的SQL方式操作。
  • Phoenix提供了批量导入/导出数据的方式。批量导入只支持csv格式,分隔符为逗号。
  • Phoenix中的SQL操作,可以马上同步到HBase,通过hbase shell检查都成功。
  • 目前Cloudera官方提供的Phoenix版本较旧,为4.7.0,社区最新版本为4.11.0
  • Phoenix提供的SQL语法较为简陋,没有insert/update,一律用upsert代替。
  • 使用upsert插入数据时,只能一条一条插入,没法将全部字段值写到一个“values”后面。

 

1、下载

https://github.com/chiastic-security/phoenix-for-cloudera/tree/4.8-HBase-1.2-cdh5.8

 

 

 

 

 

2、编译(编译时间较长,耐心等待)

mvn clean package -DskipTests

 

 

 

 

 

 

 

 

 

 

3、解压

  将编译好的phoenix-4.8.0-cdh5.8.0.tar.gz解压出来

复制代码

[root@cmbigdata1 phoenix]# tar -zxvf  phoenix-4.8.0-cdh5.8.0.tar.gz  [root@cmbigdata1 phoenix]# cd phoenix-4.8.0-cdh5.8.0  [root@cmbigdata1 phoenix-4.8.0-cdh5.8.0]# ll  total 166152  drwxr-xr-x 2 root root      4096 Apr 18 16:41 bin  -rw-r--r-- 1 root root      1930 Aug  8  2016 build.txt  drwxr-xr-x 3 root root      4096 Aug  8  2016 dev  drwxr-xr-x 2 root root      4096 Aug  8  2016 docs  drwxr-xr-x 3 root root      4096 Aug  8  2016 examples  drwxr-xr-x 2 root root      4096 Apr 18 16:40 lib  -rw-r--r-- 1 root root 113247548 Apr 18 14:43 phoenix-4.8.0-cdh5.8.0-client.jar  -rw-r--r-- 1 root root   6619716 Apr 18 14:30 phoenix-4.8.0-cdh5.8.0-queryserver.jar  -rw-r--r-- 1 root root  22498517 Apr 18 14:43 phoenix-4.8.0-cdh5.8.0-server.jar  -rw-r--r-- 1 root root  27739579 Apr 18 14:29 phoenix-4.8.0-cdh5.8.0-thin-client.jar

复制代码

 

 

 

 

 

 

4、将phoenix-4.8.0-cdh5.8.0-server.jar拷贝到每一个RegionServer下

[root@cmbigdata2~]# find / -name 'phoenix-4.8.0-cdh5.8.0-server.jar'  /soft/bigdata/clouderamanager/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/hbase/lib/phoenix-4.8.0-cdh5.8.0-server.jar

   cmbigdata2和cmbigdata3和cmbigdata4一样。

 

 

 

 

5、增加hbase-site.xml 配置

<property>  <name>hbase.table.sanity.checks</name>  <value>false</value>  </property>

 

 

 

 

 

 

 

 

 CDH修改方法:
  在集群管理页面点击Hbase,进入Hbase管理界面

 

 

 

 

 

 

点击配置:

                

 

 


选择高级:

                  

 

 


增加如下配置:

        

 

 

 

 

 

 

6、重启Hbase  

    这个很简单,不多说,会玩cloudermanager的人都知道。

7、登录phoenix

  进入phoenix-4.8.0-cdh5.8.0/bin目录执行。

 

一,Phoenix的介绍 
1,Phoenix, (“凤凰”),它相当于一个Java中间件,提供jdbc连接,操作hbase数据表。

2,Apache Phoenix是构建在HBase之上的关系型数据库层,作为内嵌的客户端JDBC驱动用以对HBase中的数据进行低延迟访问。

二,Phoenix的下载 
1,官网上下载的Phoenix的都会在文件名上标注需要搭配的hbase版本号,注意要一致。 
2,要注意在官网上http://apache.fayea.com/phoenix/ 下载,如果自己电脑上的安装的hbase版本是cdh的话,则这两者会冲突,使用sqlline.py连接hbase时候会报类似以下错误:

出错原因:phoenix官方版本pom文件里的hbase依赖并不是使用cdh版本的。 
解决的方法: 所以,为了能够使得phoenix与cdh对应,我们需要从phoenix官网下载对应版本(4.6.0)的phoenix源码,修改pom文件依赖以及部分源码,并重新编译,得到适配于cdh5.4 hbase1.0.0 的phoenix。

三,解决的步骤 
1,下载cdh版本的Phoenix,注意它需要搭配的hbase版本是hbase1.2版本。 
https://github.com/chiastic-security/phoenix-for-cloudera/tree/4.8-HBase-1.2-cdh5.8

2,然后把该文件夹(phoenix-for-cloudera-4.8-HBase-1.2-cdh5.8)拷贝解压到如下路径:

D:\Software\Phoenix\phoenix-for-cloudera-4.8-HBase-1.2-cdh5.8

3,利用maven对该文件夹(phoenix-for-cloudera-4.8-HBase-1.2-cdh5.8)进行重新编译,具体操作如下:

(1),首先电脑 要安装maven包,安装过程网上自己百度一下,不再介绍了

(2), 然后在window终端里,进入该文件夹路径下(phoenix-for-cloudera-4.8-HBase-1.2-cdh5.8):

D:\Software\Phoenix\phoenix-for-cloudera-4.8-HBase-1.2-cdh5.8>

(3),然后输入如下命令:

D:\Software\Phoenix\phoenix-for-cloudera-4.8-HBase-1.2-cdh5.8> mvn clean package -DskipTests -Dcdh.flume.version=1.6.0

(4), 最后如果显示:

则说明编译成功。

(5) 将编译打包好后的\Software\Phoenix\phoenix-for-cloudera-4.8-Hbase-1.2-cdh5.8\phoenix-assembly\target\phoenix-4.8.0-cdh5.8.0.tar.gz进行解压phoenix-4.8.0-cdh5.8.0,解压后的文件可以放在当前路径上 。

4,接下来把编译后的整个文件夹(phoenix-for-cloudera-4.8-Hbase-1.2-cdh5.8)上传到集群上。

5, 将phoenix-4.8.0-cdh5.8.0中的phoenix-4.8.0-cdh5.8.0-server.jar拷贝到每一个RegionServer下/opt/cloudera/parcels/CDH/lib/hbase/lib

6,最后一步重启hbase集群。

7,进入集群中phoenix文件夹下的bin子文件夹下输入如下命令来开启phoenix了:

./sqlline.py dsbbzx1,dsbbzx4,dsbbzx5:2181

出现如下结果: 


则说明Phoenix在集群上安装成功了,接下来就可以使用Phoenix了。
-------------------

Apache Phoenix

Phoenix supports thick and thin connection types:

Use the appropriate default.driverdefault.url, and the dependency artifact for your connection type.

Thick client connection

Properties

Name Value
default.driver org.apache.phoenix.jdbc.PhoenixDriver
default.url jdbc:phoenix:localhost:2181:/hbase-unsecure
default.user phoenix_user
default.password phoenix_password

Dependencies

Artifact Excludes
org.apache.phoenix:phoenix-core:4.4.0-HBase-1.0  

Maven Repository: org.apache.phoenix:phoenix-core

Thin client connection

Properties

Name Value
default.driver org.apache.phoenix.queryserver.client.Driver
default.url jdbc:phoenix:thin:url=http://localhost:8765;serialization=PROTOBUF
default.user phoenix_user
default.password phoenix_password

Dependencies

Before Adding one of the below dependencies, check the Phoenix version first.

Artifact Excludes Description
org.apache.phoenix:phoenix-server-client:4.7.0-HBase-1.1   For Phoenix 4.7
org.apache.phoenix:phoenix-queryserver-client:4.8.0-HBase-1.2   For Phoenix 4.8+

Maven Repository: org.apache.phoenix:phoenix-queryserver-client

详见:http://zeppelin.apache.org/docs/0.7.1/interpreter/jdbc.html#apache-phoenix

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!