Logstash - Failed to open <file_path> Permission denied

匆匆过客 提交于 2019-12-22 12:32:53

问题


I am using logstash to push all the text logs from storage to elastic search. My storage size is about 1 TB. To Start with I have started to push 368 GB data (may be few hundred thousand files) to elastic search but logstash is failing with following error.

{:timestamp=>"2014-05-15T00:41:12.436000-0700", :message=>"/root/share/archive_data/sessionLogs/965c6f46-1a5e-4820-a68d-7c32886972fc/Log.txt: file grew, old size 0, new size 1557420", :level=>:debug, :file=>"filewatch/watch.rb", :line=>"81"}
{:timestamp=>"2014-05-15T00:41:12.437000-0700", :message=>":modify for /root/share/archive_data/sessionLogs/965c6f46-1a5e-4820-a68d-7c32886972fc/Log.txt, does not exist in @files", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"77"}
{:timestamp=>"2014-05-15T00:41:12.441000-0700", :message=>"_open_file: /root/share/archive_data/sessionLogs/965c6f46-1a5e-4820-a68d-7c32886972fc/Log.txt: opening", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"98"}
{:timestamp=>"2014-05-15T00:41:12.441000-0700", :message=>"(warn supressed) failed to open /root/share/archive_data/sessionLogs/965c6f46-1a5e-4820-a68d-7c32886972fc/Log.txt: Permission denied - /root/share/archive_data/sessionLogs/965c6f46-1a5e-4820-a68d-7c32886972fc/Log.txt", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"110"}

share is network mounted. I am using root user to start logstash. User should have all the access needed to mount. share directory has following access drwxr-xr-x 44 root root 0 May 13 08:36 share

Now, my log files are static they don't change.

So, my question is - Is there anyway to let logstash know that do not store file handles once it process one log file. I think above error is occurred because number of log files is huge.

I have already filed a bug and there is existing bug in logstash which says that logstash doesn't do well when log files are more in number.

I see some duplicate issues here but I would like to know if anybody has any experience with this kind of issue?


回答1:


I think, for logstash 1.4.2, the only answer is to:

  • move or delete the files from the monitored directory
  • restart logstash

I don't think there's any other way to have logstash release file handles of logs that have been processed and won't be added to any more.



来源:https://stackoverflow.com/questions/23683268/logstash-failed-to-open-file-path-permission-denied

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!