I am looking for a functionality similar to the distributed cache of Hadoop in Spark. I need a relatively small data file (with some index values) to be present in all nodes
Please have a look at SparkContext.addFile() method. Guess that is what you were looking for.
SparkContext.addFile()