seems that there might be several reasons for getting into this "indexing" hell. I spent few hours trying to fix it using the ideas above.
At the end of the day, with some profiling work, I found that the bad guy in my case was the csv plugin:
https://plugins.jetbrains.com/plugin/10037-csv-plugin
I had few (not so large) CSV files serving as input, and although I marked them as not to be indexed, the plugin kept trying to index them.
Once I removed the plugin everything worked fine.
