flume+kafka测试(监控File)
1.flume与kafka整合 1.下载插件包 Flume和Kafka插件包下载: https://github.com/beyondj2ee/flumeng-kafka-plugin 2.复制jar包 复制插件包中的jar包到flume/lib中 (删掉不同版本相同jar包,需删除scala-compiler-z.9.2.jar包,否则flume启动会出现问题) 复制kafka/libs中的jar包到flume/lib中 2.配置 flume 配置文件(监控 file ) vi /opt/flume/conf/hw.conf agent.sources = s1 agent.channels = c1 agent.sinks = k1 agent.sources.s1.type=exec agent.sources.s1.command=tail -F /opt/log/debug.log agent.sources.s1.channels=c1 agent.channels.c1.type=memory agent.channels.c1.capacity=10000 agent.channels.c1.transactionCapacity=100 #设置Kafka接收器 agent.sinks.k1.type= org.apache.flume.sink.kafka