1.查看hdp版本,可在ambari-agent节点上查看
VERSION=`hdp-select status hadoop-client | sed 's/hadoop-client - \([0-9]\.[0-9]\).*/\1/'`
2.server节点上git克隆代码,其中下方$VERSION指的是上方的版本号
sudo git clone https://github.com/cas-bigdatalab/ambari-impala-service.git /var/lib/ambari-server/resources/stacks/HDP/$VERSION/services/IMPALA
如果出现SSL问题,可关闭SSL验证再执行
git config --global http.sslVerify false git config --global http.sslversion tlsv1
如果你没有安装git,可到https://github.com/cas-bigdatalab/ambari-impala-service地址上下载代码,放到目录/var/lib/ambari-server/resources/stacks/HDP/$VERSION/services/IMPALA下
如果你的git安装有任何问题,参考文章后方附录git的安装升级配置
3.server上目录/etc/yum.repos下新建文件impala.repo
[cloudera-cdh5] # Packages for Cloudera's Distribution for Hadoop, Version 5, on RedHat or CentOS 6 x86_64 name=Cloudera's Distribution for Hadoop, Version 5 baseurl=https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5/ gpgkey =https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera gpgcheck = 0
然后将此文件发送到各个需要安装的agent节点上
4.重启ambari-server
service ambari-server restart
5.重启成功后, 在页面里选择impala服务进行安装
这里选择server节点安装catalog , state_store
agent节点安装impala-daemon
由于要去cloudera官网下载rpm包,网速比较慢,若失败可多试几次。
6.安装成功后,在ambari服务配置里修改或添加HDFS配置
6.1 修改core-site.xml
<property> <name>dfs.client.read.shortcircuit</name> <value>true</value> </property> <property> <name>dfs.client.read.shortcircuit.skip.checksum</name> <value>false</value> </property> <property> <name>dfs.datanode.hdfs-blocks-metadata.enabled</name> <value>true</value> </property>
6.2 修改hdfs-site.xml
<property> <name>dfs.datanode.hdfs-blocks-metadata.enabled</name> <value>true</value> </property> <property> <name>dfs.block.local-path-access.user</name> <value>impala</value> </property> <property> <name>dfs.client.file-block-storage-locations.timeout.millis</name> <value>60000</value> </property>
7.重启hdfs及相关服务
8.启动impala服务
impala服务启动后一会就会停掉,不用着急,继续向下看
9.将各个节点上的hbase的jar包复制到impala目录下
需要将版本号去掉改为软连接,
因为我的节点上/usr/lib/impala/lib目录下已有hbase的三个jar包(hbase-annotations.jar,hbase-client.jar,hbase-common.jar),这几个就不需要再次建立软连接了
cp /usr/lib/hbase/lib/hbase-*.jar /usr/lib/impala/lib/ ln -s hbase-examples-1.2.0-cdh5.16.1.jar hbase-examples.jar ln -s hbase-external-blockcache-1.2.0-cdh5.16.1.jar hbase-external-blockcache.jar ln -s hbase-hadoop2-compat-1.2.0-cdh5.16.1.jar hbase-hadoop2-compat.jar ln -s hbase-hadoop-compat-1.2.0-cdh5.16.1.jar hbase-hadoop-compat.jar ln -s hbase-it-1.2.0-cdh5.16.1.jar hbase-it.jar ln -s hbase-prefix-tree-1.2.0-cdh5.16.1.jar hbase-prefix-tree.jar ln -s hbase-procedure-1.2.0-cdh5.16.1.jar hbase-procedure.jar ln -s hbase-resource-bundle-1.2.0-cdh5.16.1.jar hbase-resource-bundle.jar ln -s hbase-rest-1.2.0-cdh5.16.1.jar hbase-rest.jar ln -s hbase-rsgroup-1.2.0-cdh5.16.1.jar hbase-rsgroup-1.2.0.jar ln -s hbase-server-1.2.0-cdh5.16.1.jar hbase-server.jar ln -s hbase-shell-1.2.0-cdh5.16.1.jar hbase-shell.jar ln -s hbase-spark-1.2.0-cdh5.16.1.jar hbase-spark.jar ln -s hbase-thrift-1.2.0-cdh5.16.1.jar hbase-thrift.jar
10.错误时,查看各节点日志/var/log/impala/
我的jdk版本是1.8,并需要在各节点上修改配置文件/etc/default/bigtop-utils,修改
export JAVA_HOME=/opt/module/jdk1.8.0_112
然后执行source /etc/default/bigtop-utils
11.ambari管理页面启动impala
查看server上state_store
查看catalog
打开impalad各节点的ui
在impalad节点上打开shell客户端
参考:
https://www.cnblogs.com/zwgblog/p/6846030.html
https://github.com/cas-bigdatalab/ambari-impala-service
【附录】
git的安装升级配置
如果出现fatal: HTTP request failed此类问题,一般是由于git版本过低引起的,可参考下方升级
1.查看git版本
git --version
2.删除旧版本
yum remove git
3.安装git依赖包
yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel asciidoc yum install gcc perl-ExtUtils-MakeMaker
4.下载新版本
cd /usr/local/src/ wget https://www.kernel.org/pub/software/scm/git/git-2.15.1.tar.xz
5.解压编译
tar -vxf git-2.15.1.tar.xz cd git-2.15.1 make prefix=/usr/local/git all make prefix=/usr/local/git install echo "export PATH=$PATH:/usr/local/git/bin" >> /etc/profile source /etc/profile
6.查看版本变为2.15.1
git --version
来源:https://www.cnblogs.com/RexSheng/p/10431179.html