real time log processing using apache spark streaming

Question

I want to create a system where I can read logs in real time, and use apache spark to process it. I am confused if I should use something like kafka or flume to pass the logs to

Answer 1

Although this is a old question, posting a link from Databricks, which has a great step by step article for log analysis with Spark considering many areas.

https://databricks.gitbooks.io/databricks-spark-reference-applications/content/logs_analyzer/index.html

Hope this helps.