ambari HDP2.6.5 安装FLINK1.9

要下载Flink服务文件夹，请运行以下命令

VERSION=`hdp-select status hadoop-client | sed 's/hadoop-client - \([0-9]\.[0-9]\).*/\1/'`
sudo git clone https://github.com/abajwa-hw/ambari-flink-service.git   /var/lib/ambari-server/resources/stacks/HDP/$VERSION/services/FLINK

重新启动Ambari

ambari-server restart

flink 是1.8版本修改成1.9

修改configuration/flink-ambari-config.xml

<property>
    <name>flink_download_url</name>
    <value>http://X.X.151.15/Package/flink-1.9.0-bin-scala_2.11.tgz</value>
    <description>Snapshot download location. Downloaded when setup_prebuilt is true</description>
  </property>

修改metainfo.xml

<name>FLINK</name>
            <displayName>Flink</displayName>
            <comment>Apache Flink is a streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams.</comment>
            <version>1.9.0</version>

FLINK on YARN

flink在yarn上可以直接运行起来

<property>
	<name>yarn.client.failover-proxy-provider</name>
	<value>org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider</value>
</property>

flink在yarn上不能运行起来

<property>
	<name>yarn.client.failover-proxy-provider</name>
	<value>org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider</value>
</property>

并重启yarn集群

启动失败

Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=flink, access=EXECUTE, inode="/user/flink/.flink/application_1579166472849_0003":root:hdfs:drwxrwx---

在这里插入图片描述

解决

export HADOOP_USER_NAME=hdfs

测试flink

./flink run --jobmanager yarn-cluster -yn 1 -ytm 768 -yjm 768 /opt/flink/examples/batch/WordCount.jar --input  hdfs://test01:8020/jzytest/1.csv  --output hdfs://test01:8020/jzytest/test.txt

问题

Caused by: java.lang.ClassNotFoundException: com.sun.jersey.core.util.FeaturesAndProperties

解决

修改yarn-site.xml中的yarn.timeline-service.enabled设置为false
修改后如图

来源：CSDN

作者：jzy3711

链接：https://blog.csdn.net/jzy3711/article/details/104043860

标签

flink

ambari