channel lock error while configuring flume's multiple sources using FILE channels

问题:

Configuring multiple sources for an agent throwing me lock error using FILE channel. Below is my config file.

a1.sources = r1 r2 a1.sinks = k1 k2 a1.channels = c1 c3  #sources a1.sources.r1.type=netcat a1.sources.r1.bind=localhost a1.sources.r1.port=4444  a1.sources.r2.type=exec a1.sources.r2.command=tail -f /opt/gen_logs/logs/access.log  #sinks a1.sinks.k1.type=hdfs a1.sinks.k1.hdfs.path=/flume201 a1.sinks.k1.hdfs.filePrefix=netcat- a1.sinks.k1.rollInterval=100 a1.sinks.k1.hdfs.fileType=DataStream a1.sinks.k1.hdfs.callTimeout=100000   a1.sinks.k2.type=hdfs a1.sinks.k2.hdfs.path=/flume202 a1.sinks.k2.hdfs.filePefix=execCommand- a1.sinks.k2.rollInterval=100 a1.sinks.k2.hdfs.fileType=DataStream a1.sinks.k2.hdfs.callTimeOut=100000  #channels a1.channels.c1.type=file a1.channels.c1.checkpointDir=/home/cloudera/alpha/001 a1.channels.c3.type=file a1.channels.c3.checkpointDir=/home/cloudera/beta/001  #bind r1 c1 k1 a1.sources.r1.channels=c1 a1.sinks.k1.channel=c1  a1.sources.r2.channels=c3 a1.sinks.k2.channel=c3

I am getting below error

Channel closed [channel=c3]. Due to java.io.IOException: Cannot lock /home/cloudera/.flume/file-channel/data. The directory is already locked. [channel=c3]

But when i am using memory channel. Its working fine.

回答1:

My gues is that you need to specify different dataDirs property for each channel, cause both of them use the default value

回答2:

From the docs

By default the File Channel uses paths for checkpoint and data directories that are within the user home as specified above. As a result if you have more than one File Channel instances active within the agent, only one will be able to lock the directories and cause the other channel initialization to fail. It is therefore necessary that you provide explicit paths to all the configured channels, preferably on different disks.

The configuration for the channels should be,

#channels a1.channels.c1.type=file a1.channels.c1.checkpointDir=/home/cloudera/alpha/001 a1.channels.c1.dataDirs=/mnt/alpha_data/  a1.channels.c3.type=file a1.channels.c3.checkpointDir=/home/cloudera/beta/001 a1.channels.c3.dataDirs=/mnt/beta_data/

The dataDirs property defaults to /home/user/.flume/file-channel/data path.

转载请标明出处:channel lock error while configuring flume's multiple sources using FILE channels

文章来源: channel lock error while configuring flume's multiple sources using FILE channels